{"course":{"slug":"image-editing","title":"Editing an Image","url":"https://addaly.com/learn/image-editing","level":"beginner","track":"create"},"lesson":{"slug":"what-a-diffusion-model-does-to-your-photo","title":"Noise in, picture out: inpainting explained","url":"https://addaly.com/learn/image-editing/what-a-diffusion-model-does-to-your-photo","position":59,"of":78,"minutes":10,"answer":"Generative fill initialises a masked region with noise and denoises it towards something consistent with the surrounding pixels and your prompt, which is why filled areas are cleaner than photographs, softer than the original file.","keyIdea":"Generative fill initialises a masked region with noise and denoises it towards something consistent with the surrounding pixels and your prompt, which is why filled areas are cleaner than photographs, softer than the original file, and wrong wherever text or regular geometry is involved."},"body":{"format":"markdown","text":"## The training idea, in one paragraph\n\nA diffusion model is trained by taking real images and adding noise to them in steps until nothing remains but static. The model learns to predict, at any noise level, what noise was added, which is equivalent to learning how to remove a little of it. Run that prediction repeatedly starting from pure static and you get an image, because each step nudges the static towards something the model considers image-like.\n\nText conditioning steers the nudging. A text encoder turns your prompt into a vector, and the denoising network is trained so that its predictions depend on that vector. \"A ceramic bowl\" pushes each step towards arrangements the model associates with those words.\n\n## Inpainting: the same thing with a mask\n\nWhen you select an area and ask for generative fill, four things go in:\n\n1. **The masked region**, which is initialised with noise.\n2. **The surrounding pixels**, which are kept fixed and re-imposed at every denoising step, so the generated area is continuously pulled towards agreeing with its edges.\n3. **Your prompt**, if you gave one.\n4. **A denoising strength**, which decides how much of the original content under the mask survives. At 1.0 the region is generated from scratch; at 0.4 the original is only partially overwritten, which is useful for changing a texture while keeping a shape.\n\nMost systems work in a **latent space** rather than on pixels: an encoder compresses the image by a factor of eight in each direction, the diffusion happens in that smaller space, and a decoder expands the result back to pixels. This is why the whole thing is fast enough to be a button, and it is the source of several of the artefacts below.\n\n> Figure: What an inpaint is actually given. Steps: 1. The masked region, initialised with noise (Nothing of the original survives at full denoising strength. Around 0.4 the shape underneath still constrains the result, which is how a texture is changed and a fold kept.); 2. The surrounding pixels, re-imposed at every step (This is what pulls the generated area towards agreeing with its edges, and it is the whole reason an empty prompt continues your wall rather than inventing a generic one.); 3. Your prompt, as a vector (Short and specific beats long and elaborate. Long stacked style words dilute the conditioning rather than compounding it.); 4. Twenty to fifty denoising steps, at one-eighth scale (Diffusion happens in a latent grid eight times smaller in each direction. Fine detail is decoded rather than generated, which is the resolution ceiling.); 5. A decoded patch that is cleaner than your photograph (Likely images are not grainy, so the fill has no noise floor. Match grain over the whole picture afterwards, and expect text and brick courses to be wrong.). The surroundings are a far stronger constraint than anything you type, which is why an empty prompt is right for a removal and why a mask drawn tight to the object leaves a seam. Expand the selection by 10 to 30 pixels and feather 5 to 15, and most of the complaints about the tool disappear.\n\n## What follows from the mechanism\n\n**Generated regions are smoother and cleaner than photographs.** The model produces what it considers a likely image, and likely images are not grainy. A filled region in a high-ISO photograph is conspicuously noise-free. Add matched grain over the whole picture afterwards, as in the compositing block.\n\n**The latent grid produces a resolution ceiling.** Because diffusion happens at one-eighth scale, fine detail is decoded rather than generated, and the result is often slightly soft compared with the surrounding photograph. On a 45-megapixel file with a small fill, the difference is obvious.\n\n**Seams appear when the mask boundary is too tight.** The surrounding context is what keeps the generation consistent, and a mask drawn exactly at the edge of the object gives the model almost no overlap to work with. Expand the selection by 10 to 30 pixels and the seam usually disappears.\n\n**Text comes out wrong.** The model has learned the visual statistics of letterforms, not a writing system. A generated shopfront has letter-shaped marks that are not letters. This has improved and it has not been solved, and for anything with real text the answer is to composite the real text in.\n\n**Regular geometry drifts.** Brick courses that do not line up, window frames that change width, tiles that lose their grid. The model has no representation of a straight line continuing; it has a statistical sense of what brickwork looks like locally.\n\n**Hands, small faces and reflections are the classic failures**, all for the same reason: they are high-complexity structures that occupy few pixels, so errors there cost the training objective very little.\n\n**It is not deterministic.** The same prompt on the same mask with a different random seed gives a different result. Three attempts cost three seconds and one of them is usually much better than the others, which is why every implementation offers variations.\n\n## The controls worth learning\n\n- **Mask feather and expansion.** The single most effective control. Feather 5 to 15 pixels, expand 10 to 30.\n- **Denoising strength**, where exposed. High for replacing, low for modifying.\n- **Prompt or no prompt.** An empty prompt tells the model to continue the surroundings, which is what you want for removing an object. A described prompt is for adding something.\n- **Generate in several small passes** rather than one large one. Each pass has more surrounding context relative to its size, so each is more consistent.\n- **Seed**, where exposed, so you can reproduce a result you liked.\n\n## Where this sits against the earlier tools\n\nThe previous module's content-aware fill copies patches from your own photograph. Diffusion inpainting generates new content from a model trained on other people's photographs. The practical difference:\n\n- **Content-aware** is limited to what is in the frame, so it cannot invent a doorway, and everything it produces was in your image.\n- **Diffusion** can produce a doorway, and that doorway was never there.\n\nNeither is better in general. For removing a bin against a plain wall, content-aware is faster and safer. For removing a car from a complex street, diffusion will produce a more convincing result and will have invented the shopfront behind it.\n\n## What it does not do\n\nIt does not know what is true. The model has no representation of your client's product, this particular person's face, the actual building behind the van, or what was written on the sign. It produces the most probable-looking content given the surroundings and the prompt, and probability is not evidence.\n\nThat statement is the whole of the ethical discussion later in this module, and it is worth holding as a technical fact rather than a moral one: **the output is a sample from a distribution, not a measurement of a scene.** Everything else follows from deciding whether the picture in front of you is allowed to contain samples."},"check":{"question":"An editor removes a lamp post with generative fill, drawing the selection tightly around it. The fill matches the background but there is a visible seam. What is the most effective fix?","options":[{"text":"Raise the denoising strength to 1.0 so the region is generated entirely rather than partly derived from the lamp post.","correct":false,"tempting":"Addresses how much of the original survives inside the mask, which does not change how little surrounding context the model has to match its edges against."},{"text":"Write a more specific prompt describing the background, so the model produces content that matches more closely.","correct":false,"tempting":"Steers what is generated rather than how it joins: a well-described fill drawn tightly still meets the surroundings with almost no overlap to reconcile."},{"text":"Expand and feather the selection so the model has 10 to 30 pixels of surrounding context to reconcile the generated region against.","correct":true},{"text":"Generate at a higher output resolution, since the seam is caused by the latent space's eightfold downscaling.","correct":false,"tempting":"Names a real cause of softness inside the filled area, which is a different artefact from a discontinuity at the boundary."}]},"prev":{"slug":"matching-grain-noise-and-focus","title":"The last tell: texture and depth","url":"https://addaly.com/learn/image-editing/matching-grain-noise-and-focus"},"next":{"slug":"generative-fill-in-practice","title":"Getting a usable result in three attempts instead of thirty","url":"https://addaly.com/learn/image-editing/generative-fill-in-practice"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Noise in, picture out: inpainting explained', in Editing an Image. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/image-editing/what-a-diffusion-model-does-to-your-photo (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Noise in, picture out: inpainting explained","courseTitle":"Editing an Image","module":{"index":7,"slug":"ai-inside-the-image-editor","title":"AI inside the image editor"},"answer":null,"keyIdea":"Generative fill initialises a masked region with noise and denoises it towards something consistent with the surrounding pixels and your prompt, which is why filled areas are cleaner than photographs, softer than the original file, and wrong wherever text or regular geometry is involved.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 7.1","figure":{"type":"flow","steps":[{"label":"The masked region, initialised with noise","detail":"Nothing of the original survives at full denoising strength. Around 0.4 the shape underneath still constrains the result, which is how a texture is changed and a fold kept."},{"label":"The surrounding pixels, re-imposed at every step","detail":"This is what pulls the generated area towards agreeing with its edges, and it is the whole reason an empty prompt continues your wall rather than inventing a generic one."},{"label":"Your prompt, as a vector","detail":"Short and specific beats long and elaborate. Long stacked style words dilute the conditioning rather than compounding it."},{"label":"Twenty to fifty denoising steps, at one-eighth scale","detail":"Diffusion happens in a latent grid eight times smaller in each direction. Fine detail is decoded rather than generated, which is the resolution ceiling."},{"label":"A decoded patch that is cleaner than your photograph","detail":"Likely images are not grainy, so the fill has no noise floor. Match grain over the whole picture afterwards, and expect text and brick courses to be wrong."}],"title":"What an inpaint is actually given","caption":"The surroundings are a far stronger constraint than anything you type, which is why an empty prompt is right for a removal and why a mask drawn tight to the object leaves a seam. Expand the selection by 10 to 30 pixels and feather 5 to 15, and most of the complaints about the tool disappear."},"alt":"Figure: What an inpaint is actually given. Steps: 1. The masked region, initialised with noise (Nothing of the original survives at full denoising strength. Around 0.4 the shape underneath still constrains the result, which is how a texture is changed and a fold kept.); 2. The surrounding pixels, re-imposed at every step (This is what pulls the generated area towards agreeing with its edges, and it is the whole reason an empty prompt continues your wall rather than inventing a generic one.); 3. Your prompt, as a vector (Short and specific beats long and elaborate. Long stacked style words dilute the conditioning rather than compounding it.); 4. Twenty to fifty denoising steps, at one-eighth scale (Diffusion happens in a latent grid eight times smaller in each direction. Fine detail is decoded rather than generated, which is the resolution ceiling.); 5. A decoded patch that is cleaner than your photograph (Likely images are not grainy, so the fill has no noise floor. Match grain over the whole picture afterwards, and expect text and brick courses to be wrong.). The surroundings are a far stronger constraint than anything you type, which is why an empty prompt is right for a removal and why a mask drawn tight to the object leaves a seam. Expand the selection by 10 to 30 pixels and feather 5 to 15, and most of the complaints about the tool disappear."}},"blocks":[{"t":"h2","id":"the-training-idea-in-one-paragraph","runs":[{"text":"The training idea, in one paragraph"}]},{"t":"p","runs":[{"text":"A diffusion model is trained by taking real images and adding noise to them in steps until nothing remains but static. The model learns to predict, at any noise level, what noise was added, which is equivalent to learning how to remove a little of it. Run that prediction repeatedly starting from pure static and you get an image, because each step nudges the static towards something the model considers image-like."}]},{"t":"p","runs":[{"text":"Text conditioning steers the nudging. A text encoder turns your prompt into a vector, and the denoising network is trained so that its predictions depend on that vector. “A ceramic bowl” pushes each step towards arrangements the model associates with those words."}]},{"t":"h2","id":"inpainting-the-same-thing-with-a-mask","runs":[{"text":"Inpainting: the same thing with a mask"}]},{"t":"p","runs":[{"text":"When you select an area and ask for generative fill, four things go in:"}]},{"t":"ol","start":1,"items":[[{"text":"The masked region","b":true},{"text":", which is initialised with noise."}],[{"text":"The surrounding pixels","b":true},{"text":", which are kept fixed and re-imposed at every denoising step, so the generated area is continuously pulled towards agreeing with its edges."}],[{"text":"Your prompt","b":true},{"text":", if you gave one."}],[{"text":"A denoising strength","b":true},{"text":", which decides how much of the original content under the mask survives. At 1.0 the region is generated from scratch; at 0.4 the original is only partially overwritten, which is useful for changing a texture while keeping a shape."}]]},{"t":"p","runs":[{"text":"Most systems work in a "},{"text":"latent space","b":true},{"text":" rather than on pixels: an encoder compresses the image by a factor of eight in each direction, the diffusion happens in that smaller space, and a decoder expands the result back to pixels. This is why the whole thing is fast enough to be a button, and it is the source of several of the artefacts below."}]},{"t":"h2","id":"what-follows-from-the-mechanism","runs":[{"text":"What follows from the mechanism"}]},{"t":"p","runs":[{"text":"Generated regions are smoother and cleaner than photographs.","b":true},{"text":" The model produces what it considers a likely image, and likely images are not grainy. A filled region in a high-ISO photograph is conspicuously noise-free. Add matched grain over the whole picture afterwards, as in the compositing block."}]},{"t":"p","runs":[{"text":"The latent grid produces a resolution ceiling.","b":true},{"text":" Because diffusion happens at one-eighth scale, fine detail is decoded rather than generated, and the result is often slightly soft compared with the surrounding photograph. On a 45-megapixel file with a small fill, the difference is obvious."}]},{"t":"p","runs":[{"text":"Seams appear when the mask boundary is too tight.","b":true},{"text":" The surrounding context is what keeps the generation consistent, and a mask drawn exactly at the edge of the object gives the model almost no overlap to work with. Expand the selection by 10 to 30 pixels and the seam usually disappears."}]},{"t":"p","runs":[{"text":"Text comes out wrong.","b":true},{"text":" The model has learned the visual statistics of letterforms, not a writing system. A generated shopfront has letter-shaped marks that are not letters. This has improved and it has not been solved, and for anything with real text the answer is to composite the real text in."}]},{"t":"p","runs":[{"text":"Regular geometry drifts.","b":true},{"text":" Brick courses that do not line up, window frames that change width, tiles that lose their grid. The model has no representation of a straight line continuing; it has a statistical sense of what brickwork looks like locally."}]},{"t":"p","runs":[{"text":"Hands, small faces and reflections are the classic failures","b":true},{"text":", all for the same reason: they are high-complexity structures that occupy few pixels, so errors there cost the training objective very little."}]},{"t":"p","runs":[{"text":"It is not deterministic.","b":true},{"text":" The same prompt on the same mask with a different random seed gives a different result. Three attempts cost three seconds and one of them is usually much better than the others, which is why every implementation offers variations."}]},{"t":"h2","id":"the-controls-worth-learning","runs":[{"text":"The controls worth learning"}]},{"t":"ul","items":[[{"text":"Mask feather and expansion.","b":true},{"text":" The single most effective control. Feather 5 to 15 pixels, expand 10 to 30."}],[{"text":"Denoising strength","b":true},{"text":", where exposed. High for replacing, low for modifying."}],[{"text":"Prompt or no prompt.","b":true},{"text":" An empty prompt tells the model to continue the surroundings, which is what you want for removing an object. A described prompt is for adding something."}],[{"text":"Generate in several small passes","b":true},{"text":" rather than one large one. Each pass has more surrounding context relative to its size, so each is more consistent."}],[{"text":"Seed","b":true},{"text":", where exposed, so you can reproduce a result you liked."}]]},{"t":"h2","id":"where-this-sits-against-the-earlier-tools","runs":[{"text":"Where this sits against the earlier tools"}]},{"t":"p","runs":[{"text":"The previous module’s content-aware fill copies patches from your own photograph. Diffusion inpainting generates new content from a model trained on other people’s photographs. The practical difference:"}]},{"t":"ul","items":[[{"text":"Content-aware","b":true},{"text":" is limited to what is in the frame, so it cannot invent a doorway, and everything it produces was in your image."}],[{"text":"Diffusion","b":true},{"text":" can produce a doorway, and that doorway was never there."}]]},{"t":"p","runs":[{"text":"Neither is better in general. For removing a bin against a plain wall, content-aware is faster and safer. For removing a car from a complex street, diffusion will produce a more convincing result and will have invented the shopfront behind it."}]},{"t":"h2","id":"what-it-does-not-do","runs":[{"text":"What it does not do"}]},{"t":"p","runs":[{"text":"It does not know what is true. The model has no representation of your client’s product, this particular person’s face, the actual building behind the van, or what was written on the sign. It produces the most probable-looking content given the surroundings and the prompt, and probability is not evidence."}]},{"t":"p","runs":[{"text":"That statement is the whole of the ethical discussion later in this module, and it is worth holding as a technical fact rather than a moral one: "},{"text":"the output is a sample from a distribution, not a measurement of a scene.","b":true},{"text":" Everything else follows from deciding whether the picture in front of you is allowed to contain samples."}]}],"check":{"question":[{"text":"An editor removes a lamp post with generative fill, drawing the selection tightly around it. The fill matches the background but there is a visible seam. What is the most effective fix?"}],"options":[{"text":[{"text":"Raise the denoising strength to 1.0 so the region is generated entirely rather than partly derived from the lamp post."}],"correct":false,"tempting":[{"text":"Addresses how much of the original survives inside the mask, which does not change how little surrounding context the model has to match its edges against."}]},{"text":[{"text":"Write a more specific prompt describing the background, so the model produces content that matches more closely."}],"correct":false,"tempting":[{"text":"Steers what is generated rather than how it joins: a well-described fill drawn tightly still meets the surroundings with almost no overlap to reconcile."}]},{"text":[{"text":"Expand and feather the selection so the model has 10 to 30 pixels of surrounding context to reconcile the generated region against."}],"correct":true,"tempting":null},{"text":[{"text":"Generate at a higher output resolution, since the seam is caused by the latent space’s eightfold downscaling."}],"correct":false,"tempting":[{"text":"Names a real cause of softness inside the filled area, which is a different artefact from a discontinuity at the boundary."}]}],"order":[1,3,2,0],"why":null},"prev":{"slug":"matching-grain-noise-and-focus","title":"The last tell: texture and depth","minutes":8},"next":{"slug":"generative-fill-in-practice","title":"Getting a usable result in three attempts instead of thirty","minutes":9}}}