{"course":{"slug":"image-editing","title":"Editing an Image","url":"https://addaly.com/learn/image-editing","level":"beginner","track":"create"},"lesson":{"slug":"ai-upscaling-choices","title":"Which upscaler, for which material","url":"https://addaly.com/learn/image-editing/ai-upscaling-choices","position":62,"of":78,"minutes":9,"answer":"Upscalers come in families with different tendencies to invent texture, they must be run after denoising and before sharpening, tiling seams are fixed with a larger overlap.","keyIdea":"Upscalers come in families with different tendencies to invent texture, they must be run after denoising and before sharpening, tiling seams are fixed with a larger overlap, and 2× is close to recovery while 8× is mostly the training data rather than your photograph."},"body":{"format":"markdown","text":"## Three families, three behaviours\n\nModel-based upscalers are not one thing. Knowing which family you are using explains the result.\n\n**GAN-based super-resolution**: ESRGAN, **Real-ESRGAN**, and the many community models built on them. A generator produces a larger image and a discriminator judges whether it looks real; the generator learns to produce convincing texture. The result is sharp and detailed and the detail is invented. These are the workhorses, and Real-ESRGAN in particular was trained on synthetically degraded images so it handles real-world compression and blur rather than only clean downscales.\n\n**Transformer-based**: SwinIR and successors. Generally more faithful and less inclined to hallucinate texture, slower, and often slightly softer.\n\n**Diffusion-based**: the newest, most capable of producing convincing detail and the most likely to produce detail that has nothing to do with your image.\n\nAnd separately, **face restoration** models (GFPGAN, CodeFormer) which do not upscale generally but regenerate faces from a learned prior of what faces look like. They are frequently bundled with upscalers and they are the component that changes who somebody is.\n\n> Figure: Two families of upscaler, and what each one is for. A two by two grid, rows Real-ESRGAN, a GAN and SwinIR, a transformer, columns Texture it invents and Where it belongs: Real-ESRGAN, a GAN and Texture it invents: A great deal; Real-ESRGAN, a GAN and Where it belongs: Skin, fabric, foliage, brickwork; SwinIR, a transformer and Texture it invents: Comparatively little; SwinIR, a transformer and Where it belongs: Material that must not change. Both predict a plausible high-resolution image consistent with weak evidence. Where the evidence is strong the plausible answer is close to the true one; where it is weak the output tells you about the training set. There is no warning at the boundary, and the result looks equally confident on both sides of it.\n\n## Picking by material\n\n- **Skin, fabric, foliage, brickwork, general photographic texture.** Real-ESRGAN at 2× is excellent. This is what it was built for.\n- **Anime, illustration, flat graphics.** Use a model trained on that material: `realesrgan-x4plus-anime` and equivalents. A photographic model on line art produces mushy edges.\n- **Text, logos, signage.** Do not. The letterforms are regenerated and come back the wrong shapes. Rebuild the text as vector.\n- **Faces.** Use face restoration only when a plausible face is acceptable. On a family photograph most people are delighted. On anything identifying, it produces a different person, subtly.\n- **Heavily compressed sources.** Real-ESRGAN's degradation-aware training handles these better than most. Expect the JPEG blocks to be interpreted as content.\n- **Evidence of any kind.** No.\n\n## The order that matters\n\n**Denoise before upscaling, not after.** An upscaler trained on clean-ish inputs will interpret noise as texture and enlarge it into structured artefacts: wormy patterns, clumps, pseudo-detail. Remove the noise first and the model has signal to work from.\n\n**Do not sharpen before upscaling.** The halos get enlarged too. Sharpen at the end, at the output size, exactly as in the sharpening lesson.\n\n**Crop first.** Upscaling a 40-megapixel file to use a 600-pixel corner of it is minutes of computation thrown away.\n\n**2× twice is not the same as 4× once.** Running a 2× model twice compounds its invented detail on top of its own previous inventions, and the second pass has no real information to work with. Use the 4× model if you need 4×.\n\n## Tiling, and the seams it causes\n\nA large image does not fit in a graphics card's memory, so upscalers process it in tiles and join them. Where the tiles meet, the model had different context on each side, and a seam can appear: usually a faint line of different texture or brightness.\n\nThe fix is a **tile overlap** setting, usually called padding or tile pad. Raising it from the default 10 pixels to 32 or 64 removes most seams at a modest cost in time. In Upscayl and in the command-line `realesrgan-ncnn-vulkan` this is exposed directly.\n\nIf you see a regular grid of faint lines in an upscaled image, that is tiling, and it is fixable.\n\n## Running it free\n\n**Upscayl** is a free, open-source desktop application wrapping Real-ESRGAN and several other models, with a plain interface, batch processing and no upload. It runs on Windows, macOS and Linux, uses the GPU through Vulkan where one exists, and works on integrated graphics slowly.\n\n```bash\nrealesrgan-ncnn-vulkan -i input.jpg -o output.png -n realesrgan-x4plus -s 4 -t 256\n```\n\nThat is the command-line version, the same engine, scriptable for a batch.\n\n**chaiNNer** is a free node-based tool for building upscaling pipelines and running community models. The **OpenModelDB** collection hosts hundreds of community-trained models for specific material (old film, scanned prints, compressed video, particular game textures) and one of them is often much better on your specific material than a general model.\n\nOn the paid side, Topaz Gigapixel and Adobe's Super Resolution inside Camera Raw are convenient and, on ordinary photographic material, not dramatically better than the free options.\n\n## Checking the output\n\nCompare at 100 per cent against a bicubic enlargement of the same file. Look specifically at:\n\n- **Faces**: is this the same person?\n- **Text and numbers**: are the characters the same characters?\n- **Repeated fine texture**: fabric weave, distant foliage, a wire fence. Is the pattern the pattern that was there, or a convincing substitute?\n- **Flat areas**: has the model invented texture in what should be smooth sky or wall?\n- **Tile boundaries**: any regular grid of faint lines?\n\nIf the upscale fails any of those and the picture depends on that region, use the bicubic version, which is honest about being soft.\n\n## The limit, restated because it keeps mattering\n\nAn upscaler predicts a plausible high-resolution image consistent with the low-resolution evidence. Where the evidence is strong (a well-exposed, in-focus photograph enlarged 2×) the plausible answer is usually close to the true one and the result is useful.\n\nWhere the evidence is weak (a 200-pixel face enlarged 8×) the model is mostly drawing on its training data, and what comes out tells you about the training set rather than about your photograph. There is no warning at the boundary between those two cases, and the output looks equally confident in both.\n\nThe practical rule: **2× is usually recovery-like, 4× is interpretation, 8× is illustration.** Say which one you did."},"check":{"question":"An editor upscales a noisy ISO 6400 photograph 4× and gets wormy, clumped pseudo-detail across the shadows. What should they have done differently?","options":[{"text":"Run the 2× model twice instead of the 4× model once, so each pass had less to invent.","correct":false,"tempting":"Compounds the problem: a second pass works from the first pass's invented detail, and the noise would be enlarged twice rather than once."},{"text":"Increased the tile overlap, since the artefacts come from the model having inconsistent context between tiles.","correct":false,"tempting":"Fixes a different artefact: inconsistent tiling produces a regular grid of faint seams rather than texture distributed across the shadows."},{"text":"Denoised the file first, so the model had signal rather than noise to enlarge.","correct":true},{"text":"Used a transformer-based model, which does not hallucinate texture and would have left the noise unchanged.","correct":false,"tempting":"Picks a genuinely more faithful family, which would render the noise more faithfully rather than removing the reason the result is unusable."}]},"prev":{"slug":"generative-expand-and-outpainting","title":"Making a photograph bigger than it was taken","url":"https://addaly.com/learn/image-editing/generative-expand-and-outpainting"},"next":{"slug":"ai-denoise-and-deblur","title":"Reconstruction sold as noise reduction","url":"https://addaly.com/learn/image-editing/ai-denoise-and-deblur"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Which upscaler, for which material', in Editing an Image. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/image-editing/ai-upscaling-choices (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Which upscaler, for which material","courseTitle":"Editing an Image","module":{"index":7,"slug":"ai-inside-the-image-editor","title":"AI inside the image editor"},"answer":null,"keyIdea":"Upscalers come in families with different tendencies to invent texture, they must be run after denoising and before sharpening, tiling seams are fixed with a larger overlap, and 2× is close to recovery while 8× is mostly the training data rather than your photograph.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"cols":["Texture it invents","Where it belongs"],"rows":["Real-ESRGAN, a GAN","SwinIR, a transformer"],"type":"matrix","cells":[["A great deal","Skin, fabric, foliage, brickwork"],["Comparatively little","Material that must not change"]],"title":"Two families of upscaler, and what each one is for","caption":"Both predict a plausible high-resolution image consistent with weak evidence. Where the evidence is strong the plausible answer is close to the true one; where it is weak the output tells you about the training set. There is no warning at the boundary, and the result looks equally confident on both sides of it.","cellLabels":[["trained until a discriminator believes it","2 times is close to recovery, 4 times is interpretation, 8 times is illustration"],["slower, and often slightly softer","and none of it is evidence of anything"]]},"alt":"Figure: Two families of upscaler, and what each one is for. A two by two grid, rows Real-ESRGAN, a GAN and SwinIR, a transformer, columns Texture it invents and Where it belongs: Real-ESRGAN, a GAN and Texture it invents: A great deal; Real-ESRGAN, a GAN and Where it belongs: Skin, fabric, foliage, brickwork; SwinIR, a transformer and Texture it invents: Comparatively little; SwinIR, a transformer and Where it belongs: Material that must not change. Both predict a plausible high-resolution image consistent with weak evidence. Where the evidence is strong the plausible answer is close to the true one; where it is weak the output tells you about the training set. There is no warning at the boundary, and the result looks equally confident on both sides of it."}},"blocks":[{"t":"h2","id":"three-families-three-behaviours","runs":[{"text":"Three families, three behaviours"}]},{"t":"p","runs":[{"text":"Model-based upscalers are not one thing. Knowing which family you are using explains the result."}]},{"t":"p","runs":[{"text":"GAN-based super-resolution","b":true},{"text":": ESRGAN, "},{"text":"Real-ESRGAN","b":true},{"text":", and the many community models built on them. A generator produces a larger image and a discriminator judges whether it looks real; the generator learns to produce convincing texture. The result is sharp and detailed and the detail is invented. These are the workhorses, and Real-ESRGAN in particular was trained on synthetically degraded images so it handles real-world compression and blur rather than only clean downscales."}]},{"t":"p","runs":[{"text":"Transformer-based","b":true},{"text":": SwinIR and successors. Generally more faithful and less inclined to hallucinate texture, slower, and often slightly softer."}]},{"t":"p","runs":[{"text":"Diffusion-based","b":true},{"text":": the newest, most capable of producing convincing detail and the most likely to produce detail that has nothing to do with your image."}]},{"t":"p","runs":[{"text":"And separately, "},{"text":"face restoration","b":true},{"text":" models (GFPGAN, CodeFormer) which do not upscale generally but regenerate faces from a learned prior of what faces look like. They are frequently bundled with upscalers and they are the component that changes who somebody is."}]},{"t":"h2","id":"picking-by-material","runs":[{"text":"Picking by material"}]},{"t":"ul","items":[[{"text":"Skin, fabric, foliage, brickwork, general photographic texture.","b":true},{"text":" Real-ESRGAN at 2× is excellent. This is what it was built for."}],[{"text":"Anime, illustration, flat graphics.","b":true},{"text":" Use a model trained on that material: "},{"text":"realesrgan-x4plus-anime","code":true},{"text":" and equivalents. A photographic model on line art produces mushy edges."}],[{"text":"Text, logos, signage.","b":true},{"text":" Do not. The letterforms are regenerated and come back the wrong shapes. Rebuild the text as vector."}],[{"text":"Faces.","b":true},{"text":" Use face restoration only when a plausible face is acceptable. On a family photograph most people are delighted. On anything identifying, it produces a different person, subtly."}],[{"text":"Heavily compressed sources.","b":true},{"text":" Real-ESRGAN’s degradation-aware training handles these better than most. Expect the JPEG blocks to be interpreted as content."}],[{"text":"Evidence of any kind.","b":true},{"text":" No."}]]},{"t":"h2","id":"the-order-that-matters","runs":[{"text":"The order that matters"}]},{"t":"p","runs":[{"text":"Denoise before upscaling, not after.","b":true},{"text":" An upscaler trained on clean-ish inputs will interpret noise as texture and enlarge it into structured artefacts: wormy patterns, clumps, pseudo-detail. Remove the noise first and the model has signal to work from."}]},{"t":"p","runs":[{"text":"Do not sharpen before upscaling.","b":true},{"text":" The halos get enlarged too. Sharpen at the end, at the output size, exactly as in the sharpening lesson."}]},{"t":"p","runs":[{"text":"Crop first.","b":true},{"text":" Upscaling a 40-megapixel file to use a 600-pixel corner of it is minutes of computation thrown away."}]},{"t":"p","runs":[{"text":"2× twice is not the same as 4× once.","b":true},{"text":" Running a 2× model twice compounds its invented detail on top of its own previous inventions, and the second pass has no real information to work with. Use the 4× model if you need 4×."}]},{"t":"h2","id":"tiling-and-the-seams-it-causes","runs":[{"text":"Tiling, and the seams it causes"}]},{"t":"p","runs":[{"text":"A large image does not fit in a graphics card’s memory, so upscalers process it in tiles and join them. Where the tiles meet, the model had different context on each side, and a seam can appear: usually a faint line of different texture or brightness."}]},{"t":"p","runs":[{"text":"The fix is a "},{"text":"tile overlap","b":true},{"text":" setting, usually called padding or tile pad. Raising it from the default 10 pixels to 32 or 64 removes most seams at a modest cost in time. In Upscayl and in the command-line "},{"text":"realesrgan-ncnn-vulkan","code":true},{"text":" this is exposed directly."}]},{"t":"p","runs":[{"text":"If you see a regular grid of faint lines in an upscaled image, that is tiling, and it is fixable."}]},{"t":"h2","id":"running-it-free","runs":[{"text":"Running it free"}]},{"t":"p","runs":[{"text":"Upscayl","b":true},{"text":" is a free, open-source desktop application wrapping Real-ESRGAN and several other models, with a plain interface, batch processing and no upload. It runs on Windows, macOS and Linux, uses the GPU through Vulkan where one exists, and works on integrated graphics slowly."}]},{"t":"code","lang":"bash","label":"Bash","text":"realesrgan-ncnn-vulkan -i input.jpg -o output.png -n realesrgan-x4plus -s 4 -t 256"},{"t":"p","runs":[{"text":"That is the command-line version, the same engine, scriptable for a batch."}]},{"t":"p","runs":[{"text":"chaiNNer","b":true},{"text":" is a free node-based tool for building upscaling pipelines and running community models. The "},{"text":"OpenModelDB","b":true},{"text":" collection hosts hundreds of community-trained models for specific material (old film, scanned prints, compressed video, particular game textures) and one of them is often much better on your specific material than a general model."}]},{"t":"p","runs":[{"text":"On the paid side, Topaz Gigapixel and Adobe’s Super Resolution inside Camera Raw are convenient and, on ordinary photographic material, not dramatically better than the free options."}]},{"t":"h2","id":"checking-the-output","runs":[{"text":"Checking the output"}]},{"t":"p","runs":[{"text":"Compare at 100 per cent against a bicubic enlargement of the same file. Look specifically at:"}]},{"t":"ul","items":[[{"text":"Faces","b":true},{"text":": is this the same person?"}],[{"text":"Text and numbers","b":true},{"text":": are the characters the same characters?"}],[{"text":"Repeated fine texture","b":true},{"text":": fabric weave, distant foliage, a wire fence. Is the pattern the pattern that was there, or a convincing substitute?"}],[{"text":"Flat areas","b":true},{"text":": has the model invented texture in what should be smooth sky or wall?"}],[{"text":"Tile boundaries","b":true},{"text":": any regular grid of faint lines?"}]]},{"t":"p","runs":[{"text":"If the upscale fails any of those and the picture depends on that region, use the bicubic version, which is honest about being soft."}]},{"t":"h2","id":"the-limit-restated-because-it-keeps-mattering","runs":[{"text":"The limit, restated because it keeps mattering"}],"role":"limit"},{"t":"p","runs":[{"text":"An upscaler predicts a plausible high-resolution image consistent with the low-resolution evidence. Where the evidence is strong (a well-exposed, in-focus photograph enlarged 2×) the plausible answer is usually close to the true one and the result is useful."}]},{"t":"p","runs":[{"text":"Where the evidence is weak (a 200-pixel face enlarged 8×) the model is mostly drawing on its training data, and what comes out tells you about the training set rather than about your photograph. There is no warning at the boundary between those two cases, and the output looks equally confident in both."}]},{"t":"p","runs":[{"text":"The practical rule: "},{"text":"2× is usually recovery-like, 4× is interpretation, 8× is illustration.","b":true},{"text":" Say which one you did."}]}],"check":{"question":[{"text":"An editor upscales a noisy ISO 6400 photograph 4× and gets wormy, clumped pseudo-detail across the shadows. What should they have done differently?"}],"options":[{"text":[{"text":"Run the 2× model twice instead of the 4× model once, so each pass had less to invent."}],"correct":false,"tempting":[{"text":"Compounds the problem: a second pass works from the first pass’s invented detail, and the noise would be enlarged twice rather than once."}]},{"text":[{"text":"Increased the tile overlap, since the artefacts come from the model having inconsistent context between tiles."}],"correct":false,"tempting":[{"text":"Fixes a different artefact: inconsistent tiling produces a regular grid of faint seams rather than texture distributed across the shadows."}]},{"text":[{"text":"Denoised the file first, so the model had signal rather than noise to enlarge."}],"correct":true,"tempting":null},{"text":[{"text":"Used a transformer-based model, which does not hallucinate texture and would have left the noise unchanged."}],"correct":false,"tempting":[{"text":"Picks a genuinely more faithful family, which would render the noise more faithfully rather than removing the reason the result is unusable."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"generative-expand-and-outpainting","title":"Making a photograph bigger than it was taken","minutes":8},"next":{"slug":"ai-denoise-and-deblur","title":"Reconstruction sold as noise reduction","minutes":9}}}