{"course":{"slug":"making-things-with-ai","title":"Making Things with AI","url":"https://addaly.com/learn/making-things-with-ai","level":"beginner","track":"create"},"lesson":{"slug":"what-the-model-is-doing","title":"What the model is actually doing","url":"https://addaly.com/learn/making-things-with-ai/what-the-model-is-doing","position":1,"of":84,"minutes":7,"answer":"An image model does not draw. It removes noise from static, over and over, steered by your words.","keyIdea":"An image model does not draw. It removes noise from static, over and over, steered by your words."},"body":{"format":"markdown","text":"## Start with a photograph you ruin\n\nTake a photo. Add a little random speckle. Add a little more. Keep going for a thousand rounds and you have television static: the photo is gone. That process is easy and needs no intelligence at all.\n\nHere is the trick the whole field is built on: that ruining is reversible, if you can predict, for a slightly noisy image, exactly which speckle was added.\n\n## What training actually does\n\nTraining a diffusion model is boring in a good way. Take an image from the training set. Pick a random amount of noise. Add it. Show the model the noisy image and tell it how much noise is in there. The model guesses what noise was added. Compare with the real answer, nudge the weights, repeat a few hundred million times.\n\nThat is it. The model never learns \"what a cat looks like\" as a fact you could look up. It learns one narrow skill: given a mess, guess which part of it is mess.\n\n## Generating is that skill run backwards\n\nStart with pure static: random numbers produced from a seed you can write down. Ask the model what noise it sees. Subtract a fraction of it. Ask again. After 20 to 50 rounds, static has become a picture, because at every step the model pushed the image slightly toward \"looks like the training photos\".\n\nTwo things follow immediately.\n\n- The seed decides the starting static. The same seed with the same prompt gives you the same image every time. A different seed gives a different picture from identical words.\n- The model has no canvas, no layers, no plan. Composition settles in the first handful of steps and gets refined afterwards. That is why changing one word can move the whole layout.\n\n## Where your words come in\n\nA text encoder turns your prompt into a list of numbers. Those numbers are fed into the denoiser at every step, so its guess becomes \"what noise is here, given that this is supposed to be a wet street at night\".\n\nThen a technique called classifier-free guidance sharpens it. The model runs twice per step (once with your prompt, once with nothing) and the difference between the two is amplified. How much you amplify is the guidance scale, usually somewhere around 3 to 8. Set it low and the image drifts off your prompt. Set it high and you get the burnt, over-saturated, over-contrasted look people recognise as \"an AI image\". That look is not a style. It is a knob turned too far.\n\n## Doing it in a smaller room\n\nDenoising a 1024x1024 image directly means handling over three million numbers on every step. So most models work in a compressed space. A separate autoencoder squashes the image roughly eight times in each direction: 1024x1024 becomes something like 128x128 with a few channels, about 48 times fewer numbers. All the denoising happens there, and a decoder expands the result back into pixels at the end.\n\n> Figure: One image, from a seed to a file. Steps: 1. Seed (A number you can write down. It fixes every value in the starting static, and it is what decides the composition.); 2. Text encoder (Your prompt becomes a sequence of vectors. The denoiser looks at that sequence at every step and at every position in the grid.); 3. Denoise, twenty to fifty times (Predict the noise in the current latent, subtract a fraction, move down the schedule. Composition settles in the first few steps.); 4. Guidance, twice per step (The model runs once with your prompt and once with nothing, and the result travels past the difference. This is why generation costs two forward passes.); 5. Decoder (The 128 by 128 latent grid is expanded back into a 1024 by 1024 picture. Anything too small to be held in the latent returns as a smear.); 6. PNG on disk (Most local tools write the whole recipe into a text chunk in the file. Export to JPEG and the record is gone.). Nothing here is a plan or a canvas. That is why changing one word can move the whole layout, and why rerolling the seed is often the fix rather than another adjective.\n\nThat compression is why very fine detail comes back mushy: a face forty pixels tall, small lettering, thin wires. It was never represented finely enough to survive the round trip.\n\n## Not every model works this way\n\nNewer image models swap the original network for a transformer, and swap noise prediction for flow matching, which learns a straighter path from noise to image and needs fewer steps. Some systems generate images the way a chat model generates text, one token at a time, which handles instructions and lettering better and behaves differently when you edit. The mental model above still gets you most of the way, but do not assume every tool has a seed field or a guidance slider.\n\n## Why bother knowing this\n\nBecause it changes what you do when something fails. If the model has no plan, then \"be more specific\" is often not the fix and rerolling the seed is. If detail was destroyed by compression, no number of adjectives recovers a tiny face, but regenerating that region larger will. Most prompt folklore is people mistaking mechanics for magic."},"check":{"question":"You generate an image with a fixed seed and like everything except one background element. You change a single word in the prompt and generate again with the same seed. What should you expect?","options":[{"text":"Only the part of the picture described by the changed word moves, because the seed pins the rest of the image in place.","correct":false,"tempting":"Treats the seed as fixing the content of the image rather than only the random starting noise."},{"text":"Nothing changes, because with the seed held constant the output is fully determined.","correct":false,"tempting":"Assumes the seed alone determines the result, forgetting the prompt steers every step."},{"text":"The whole image can shift noticeably, because the prompt steers every denoising step including the earliest ones, where composition is decided.","correct":true},{"text":"Only fine detail changes, because the early steps are governed by the seed and the prompt only influences the final steps.","correct":false,"tempting":"Imagines the noise already contains a layout that the prompt merely decorates at the end."}]},"prev":null,"next":{"slug":"noise-and-the-schedule","title":"Noise, steps and the schedule","url":"https://addaly.com/learn/making-things-with-ai/noise-and-the-schedule"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What the model is actually doing', in Making Things with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/making-things-with-ai/what-the-model-is-doing (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What the model is actually doing","courseTitle":"Making Things with AI","module":{"index":1,"slug":"how-the-picture-gets-made","title":"How the picture gets made"},"answer":null,"keyIdea":"An image model does not draw. It removes noise from static, over and over, steered by your words.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 1.1","figure":{"type":"flow","steps":[{"label":"Seed","detail":"A number you can write down. It fixes every value in the starting static, and it is what decides the composition."},{"label":"Text encoder","detail":"Your prompt becomes a sequence of vectors. The denoiser looks at that sequence at every step and at every position in the grid."},{"label":"Denoise, twenty to fifty times","detail":"Predict the noise in the current latent, subtract a fraction, move down the schedule. Composition settles in the first few steps."},{"label":"Guidance, twice per step","detail":"The model runs once with your prompt and once with nothing, and the result travels past the difference. This is why generation costs two forward passes."},{"label":"Decoder","detail":"The 128 by 128 latent grid is expanded back into a 1024 by 1024 picture. Anything too small to be held in the latent returns as a smear."},{"label":"PNG on disk","detail":"Most local tools write the whole recipe into a text chunk in the file. Export to JPEG and the record is gone."}],"title":"One image, from a seed to a file","caption":"Nothing here is a plan or a canvas. That is why changing one word can move the whole layout, and why rerolling the seed is often the fix rather than another adjective."},"alt":"Figure: One image, from a seed to a file. Steps: 1. Seed (A number you can write down. It fixes every value in the starting static, and it is what decides the composition.); 2. Text encoder (Your prompt becomes a sequence of vectors. The denoiser looks at that sequence at every step and at every position in the grid.); 3. Denoise, twenty to fifty times (Predict the noise in the current latent, subtract a fraction, move down the schedule. Composition settles in the first few steps.); 4. Guidance, twice per step (The model runs once with your prompt and once with nothing, and the result travels past the difference. This is why generation costs two forward passes.); 5. Decoder (The 128 by 128 latent grid is expanded back into a 1024 by 1024 picture. Anything too small to be held in the latent returns as a smear.); 6. PNG on disk (Most local tools write the whole recipe into a text chunk in the file. Export to JPEG and the record is gone.). Nothing here is a plan or a canvas. That is why changing one word can move the whole layout, and why rerolling the seed is often the fix rather than another adjective."}},"blocks":[{"t":"h2","id":"start-with-a-photograph-you-ruin","runs":[{"text":"Start with a photograph you ruin"}]},{"t":"p","runs":[{"text":"Take a photo. Add a little random speckle. Add a little more. Keep going for a thousand rounds and you have television static: the photo is gone. That process is easy and needs no intelligence at all."}]},{"t":"p","runs":[{"text":"Here is the trick the whole field is built on: that ruining is reversible, if you can predict, for a slightly noisy image, exactly which speckle was added."}]},{"t":"h2","id":"what-training-actually-does","runs":[{"text":"What training actually does"}]},{"t":"p","runs":[{"text":"Training a diffusion model is boring in a good way. Take an image from the training set. Pick a random amount of noise. Add it. Show the model the noisy image and tell it how much noise is in there. The model guesses what noise was added. Compare with the real answer, nudge the weights, repeat a few hundred million times."}]},{"t":"p","runs":[{"text":"That is it. The model never learns “what a cat looks like” as a fact you could look up. It learns one narrow skill: given a mess, guess which part of it is mess."}]},{"t":"h2","id":"generating-is-that-skill-run-backwards","runs":[{"text":"Generating is that skill run backwards"}]},{"t":"p","runs":[{"text":"Start with pure static: random numbers produced from a seed you can write down. Ask the model what noise it sees. Subtract a fraction of it. Ask again. After 20 to 50 rounds, static has become a picture, because at every step the model pushed the image slightly toward “looks like the training photos”."}]},{"t":"p","runs":[{"text":"Two things follow immediately."}]},{"t":"ul","items":[[{"text":"The seed decides the starting static. The same seed with the same prompt gives you the same image every time. A different seed gives a different picture from identical words."}],[{"text":"The model has no canvas, no layers, no plan. Composition settles in the first handful of steps and gets refined afterwards. That is why changing one word can move the whole layout."}]]},{"t":"h2","id":"where-your-words-come-in","runs":[{"text":"Where your words come in"}]},{"t":"p","runs":[{"text":"A text encoder turns your prompt into a list of numbers. Those numbers are fed into the denoiser at every step, so its guess becomes “what noise is here, given that this is supposed to be a wet street at night”."}]},{"t":"p","runs":[{"text":"Then a technique called classifier-free guidance sharpens it. The model runs twice per step (once with your prompt, once with nothing) and the difference between the two is amplified. How much you amplify is the guidance scale, usually somewhere around 3 to 8. Set it low and the image drifts off your prompt. Set it high and you get the burnt, over-saturated, over-contrasted look people recognise as “an AI image”. That look is not a style. It is a knob turned too far."}]},{"t":"h2","id":"doing-it-in-a-smaller-room","runs":[{"text":"Doing it in a smaller room"}]},{"t":"p","runs":[{"text":"Denoising a 1024x1024 image directly means handling over three million numbers on every step. So most models work in a compressed space. A separate autoencoder squashes the image roughly eight times in each direction: 1024x1024 becomes something like 128x128 with a few channels, about 48 times fewer numbers. All the denoising happens there, and a decoder expands the result back into pixels at the end."}]},{"t":"p","runs":[{"text":"That compression is why very fine detail comes back mushy: a face forty pixels tall, small lettering, thin wires. It was never represented finely enough to survive the round trip."}]},{"t":"h2","id":"not-every-model-works-this-way","runs":[{"text":"Not every model works this way"}]},{"t":"p","runs":[{"text":"Newer image models swap the original network for a transformer, and swap noise prediction for flow matching, which learns a straighter path from noise to image and needs fewer steps. Some systems generate images the way a chat model generates text, one token at a time, which handles instructions and lettering better and behaves differently when you edit. The mental model above still gets you most of the way, but do not assume every tool has a seed field or a guidance slider."}]},{"t":"h2","id":"why-bother-knowing-this","runs":[{"text":"Why bother knowing this"}]},{"t":"p","runs":[{"text":"Because it changes what you do when something fails. If the model has no plan, then “be more specific” is often not the fix and rerolling the seed is. If detail was destroyed by compression, no number of adjectives recovers a tiny face, but regenerating that region larger will. Most prompt folklore is people mistaking mechanics for magic."}]}],"check":{"question":[{"text":"You generate an image with a fixed seed and like everything except one background element. You change a single word in the prompt and generate again with the same seed. What should you expect?"}],"options":[{"text":[{"text":"Only the part of the picture described by the changed word moves, because the seed pins the rest of the image in place."}],"correct":false,"tempting":[{"text":"Treats the seed as fixing the content of the image rather than only the random starting noise."}]},{"text":[{"text":"Nothing changes, because with the seed held constant the output is fully determined."}],"correct":false,"tempting":[{"text":"Assumes the seed alone determines the result, forgetting the prompt steers every step."}]},{"text":[{"text":"The whole image can shift noticeably, because the prompt steers every denoising step including the earliest ones, where composition is decided."}],"correct":true,"tempting":null},{"text":[{"text":"Only fine detail changes, because the early steps are governed by the seed and the prompt only influences the final steps."}],"correct":false,"tempting":[{"text":"Imagines the noise already contains a layout that the prompt merely decorates at the end."}]}],"order":[1,0,2,3],"why":null},"prev":null,"next":{"slug":"noise-and-the-schedule","title":"Noise, steps and the schedule","minutes":9}}}