{"course":{"slug":"ai-images","title":"Image Generation, in Practice","url":"https://addaly.com/learn/ai-images","level":"beginner","track":"create"},"lesson":{"slug":"telling-it-the-composition","title":"The model is guessing your composition","url":"https://addaly.com/learn/ai-images/telling-it-the-composition","position":33,"of":69,"minutes":9,"answer":"Conditioning makes composition an input rather than an outcome, and letting the control end around 60% of the steps is what stops the result looking traced.","keyIdea":"Conditioning makes composition an input rather than an outcome, and letting the control end around 60% of the steps is what stops the result looking traced."},"body":{"format":"markdown","text":"## Every generation is a guess about where things go\n\nThe prompt says a woman reaching upward for a book on a high shelf. The model produces a woman, a shelf and some books, and arranges them however the training data suggests. Run it again and the arrangement changes. You are sampling from a distribution of plausible compositions, not specifying one.\n\nConditioning changes that. Instead of hoping, you hand the model the composition directly.\n\n## The mechanism, in one paragraph\n\nTake a reference image and run it through a preprocessor that throws away almost everything and keeps one kind of structure: a stick figure of the joints, a greyscale near-and-far map, a map of edges. A second network reads that structure and, at every denoising step, nudges the image toward matching it. The prompt still decides what things look like. The map decides where they are.\n\nThis is the ControlNet family, released in 2023 by Lvmin Zhang, and it is the single biggest reason working practitioners use ComfyUI or Forge rather than a hosted box.\n\n## Which map for which job\n\n- **OpenPose**: a skeleton of body, hand and face keypoints. Use it when the pose is the thing and nothing else about the reference should carry over. You can pose a stick figure by hand and generate from that, with no source photograph at all.\n- **Depth**: a greyscale image where near is bright and far is dark. Use it to keep a three-dimensional layout while changing everything's appearance: same room, different decade. The best choice for architecture, interiors and putting a product into a scene.\n- **Canny**: a strict edge map. Keeps every line it finds. Use for turning a clean line drawing into a rendered image.\n- **Lineart** and **Softedge**: looser edge detection. Softedge in particular gives the model room to reinterpret while keeping the shape.\n- **Scribble**: deliberately crude. Draw with a mouse in thirty seconds, get a composition.\n- **Segmentation**: regions labelled by category: sky here, building there, road there.\n\n## The two settings that decide whether it looks alive\n\n**Control weight.** At 1.0 the conditioning is enforced hard. Drop to 0.6-0.8 and the model has room to make the image work as an image.\n\n**Start and end percent.** This is the one people miss. You can apply the control for only part of the denoising process. Composition is decided early; detail and texture are decided late. So run the control from 0% to about 60% of the steps and then let go. The layout is locked in by then, and the model spends the final steps resolving the picture naturally instead of tracing.\n\nAn image conditioned at weight 1.0 for 100% of the steps looks stiff, flat and traced, because it is. Ending the control early is the difference between a render and a colouring-in exercise.\n\n## The failure modes\n\n**Stacked controls fighting each other.** Pose plus depth plus canny, all at full weight, leaves the model no freedom and no way to satisfy all three. The result is muddy and rigid. Use one control. Add a second only when you can name what it is contributing.\n\n**Canny eating artefacts.** Canny finds edges, and JPEG compression blocks, paper grain and scanner noise all look like edges. Run canny on a compressed photo and you get thin phantom lines in the sky. Raise the canny thresholds, or clean the source first, or use softedge instead.\n\n**Depth flattening.** Depth maps from a photo with a plain background often read the whole background as one distance, so your generated scene has a wall where you wanted a street.\n\n## Where hosted tools stand\n\nSome expose a sketch-to-image or pose-reference feature that is conditioning under a friendlier name. Most do not expose it at all, and none of them give you the weight and scheduling controls. If your work needs a pose held across a series, this is usually the moment you install something local.\n\n## The free path is the whole path\n\nThere is no paid version of this that is better. The ControlNet models are free downloads on Hugging Face. **ComfyUI** and **Forge** expose every setting. **Krita with AI Diffusion** exposes them in a way that will not frighten you, with a canvas you can draw the scribble on directly. Free GPU hours on Colab or Kaggle run all of it. See [running models yourself](/learn/running-models-yourself).\n\nToday: draw a stick figure or a five-line scribble, run it through scribble or openpose conditioning at weight 0.8 ending at 60%, and watch a composition you chose come back rendered. That is the moment most people stop treating generation as a slot machine."},"check":{"question":"A designer conditions a render on a scanned pencil sketch using Canny edges at weight 1.0 for the full run. The result looks stiff and flat, and there are thin stray lines across the sky. What went wrong?","options":[{"text":"Canny is the wrong preprocessor for drawings and depth conditioning should be used instead.","correct":false,"tempting":"Swaps the tool when the settings, not the choice of tool, produced both problems."},{"text":"Canny keeps every edge it detects, including paper grain and scan artefacts, and holding the control at full weight for all the steps prevents the model from resolving the drawing into a rendered image; raising the thresholds and ending the control around 60% fixes both.","correct":true},{"text":"The sketch was scanned at too high a resolution, so far too many edges were detected.","correct":false,"tempting":"Blames the input's size for a problem caused by edge thresholds and control scheduling."},{"text":"The run needs more denoising steps so the model has time to smooth the traced lines away.","correct":false,"tempting":"Expects extra compute to overcome a constraint that is being re-applied at every one of those steps."}]},"prev":{"slug":"show-dont-describe","title":"Stop describing it and show it the picture","url":"https://addaly.com/learn/ai-images/show-dont-describe"},"next":{"slug":"the-img2img-ladder","title":"Three passes down the denoise ladder","url":"https://addaly.com/learn/ai-images/the-img2img-ladder"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The model is guessing your composition', in Image Generation, in Practice. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-images/telling-it-the-composition (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The model is guessing your composition","courseTitle":"Image Generation, in Practice","module":{"index":4,"slug":"steering-with-images","title":"Steering with pictures instead of adjectives"},"answer":null,"keyIdea":"Conditioning makes composition an input rather than an outcome, and letting the control end around 60% of the steps is what stops the result looking traced.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"every-generation-is-a-guess-about-where-things-go","runs":[{"text":"Every generation is a guess about where things go"}]},{"t":"p","runs":[{"text":"The prompt says a woman reaching upward for a book on a high shelf. The model produces a woman, a shelf and some books, and arranges them however the training data suggests. Run it again and the arrangement changes. You are sampling from a distribution of plausible compositions, not specifying one."}]},{"t":"p","runs":[{"text":"Conditioning changes that. Instead of hoping, you hand the model the composition directly."}]},{"t":"h2","id":"the-mechanism-in-one-paragraph","runs":[{"text":"The mechanism, in one paragraph"}]},{"t":"p","runs":[{"text":"Take a reference image and run it through a preprocessor that throws away almost everything and keeps one kind of structure: a stick figure of the joints, a greyscale near-and-far map, a map of edges. A second network reads that structure and, at every denoising step, nudges the image toward matching it. The prompt still decides what things look like. The map decides where they are."}]},{"t":"p","runs":[{"text":"This is the ControlNet family, released in 2023 by Lvmin Zhang, and it is the single biggest reason working practitioners use ComfyUI or Forge rather than a hosted box."}]},{"t":"h2","id":"which-map-for-which-job","runs":[{"text":"Which map for which job"}]},{"t":"ul","items":[[{"text":"OpenPose","b":true},{"text":": a skeleton of body, hand and face keypoints. Use it when the pose is the thing and nothing else about the reference should carry over. You can pose a stick figure by hand and generate from that, with no source photograph at all."}],[{"text":"Depth","b":true},{"text":": a greyscale image where near is bright and far is dark. Use it to keep a three-dimensional layout while changing everything’s appearance: same room, different decade. The best choice for architecture, interiors and putting a product into a scene."}],[{"text":"Canny","b":true},{"text":": a strict edge map. Keeps every line it finds. Use for turning a clean line drawing into a rendered image."}],[{"text":"Lineart","b":true},{"text":" and "},{"text":"Softedge","b":true},{"text":": looser edge detection. Softedge in particular gives the model room to reinterpret while keeping the shape."}],[{"text":"Scribble","b":true},{"text":": deliberately crude. Draw with a mouse in thirty seconds, get a composition."}],[{"text":"Segmentation","b":true},{"text":": regions labelled by category: sky here, building there, road there."}]]},{"t":"h2","id":"the-two-settings-that-decide-whether-it-looks-alive","runs":[{"text":"The two settings that decide whether it looks alive"}]},{"t":"p","runs":[{"text":"Control weight.","b":true},{"text":" At 1.0 the conditioning is enforced hard. Drop to 0.6-0.8 and the model has room to make the image work as an image."}]},{"t":"p","runs":[{"text":"Start and end percent.","b":true},{"text":" This is the one people miss. You can apply the control for only part of the denoising process. Composition is decided early; detail and texture are decided late. So run the control from 0% to about 60% of the steps and then let go. The layout is locked in by then, and the model spends the final steps resolving the picture naturally instead of tracing."}]},{"t":"p","runs":[{"text":"An image conditioned at weight 1.0 for 100% of the steps looks stiff, flat and traced, because it is. Ending the control early is the difference between a render and a colouring-in exercise."}]},{"t":"h2","id":"the-failure-modes","runs":[{"text":"The failure modes"}]},{"t":"p","runs":[{"text":"Stacked controls fighting each other.","b":true},{"text":" Pose plus depth plus canny, all at full weight, leaves the model no freedom and no way to satisfy all three. The result is muddy and rigid. Use one control. Add a second only when you can name what it is contributing."}]},{"t":"p","runs":[{"text":"Canny eating artefacts.","b":true},{"text":" Canny finds edges, and JPEG compression blocks, paper grain and scanner noise all look like edges. Run canny on a compressed photo and you get thin phantom lines in the sky. Raise the canny thresholds, or clean the source first, or use softedge instead."}]},{"t":"p","runs":[{"text":"Depth flattening.","b":true},{"text":" Depth maps from a photo with a plain background often read the whole background as one distance, so your generated scene has a wall where you wanted a street."}]},{"t":"h2","id":"where-hosted-tools-stand","runs":[{"text":"Where hosted tools stand"}]},{"t":"p","runs":[{"text":"Some expose a sketch-to-image or pose-reference feature that is conditioning under a friendlier name. Most do not expose it at all, and none of them give you the weight and scheduling controls. If your work needs a pose held across a series, this is usually the moment you install something local."}]},{"t":"h2","id":"the-free-path-is-the-whole-path","runs":[{"text":"The free path is the whole path"}],"role":"free"},{"t":"p","runs":[{"text":"There is no paid version of this that is better. The ControlNet models are free downloads on Hugging Face. "},{"text":"ComfyUI","b":true},{"text":" and "},{"text":"Forge","b":true},{"text":" expose every setting. "},{"text":"Krita with AI Diffusion","b":true},{"text":" exposes them in a way that will not frighten you, with a canvas you can draw the scribble on directly. Free GPU hours on Colab or Kaggle run all of it. See "},{"text":"running models yourself","href":"https://addaly.com/learn/running-models-yourself"},{"text":"."}]},{"t":"p","runs":[{"text":"Today: draw a stick figure or a five-line scribble, run it through scribble or openpose conditioning at weight 0.8 ending at 60%, and watch a composition you chose come back rendered. That is the moment most people stop treating generation as a slot machine."}]}],"check":{"question":[{"text":"A designer conditions a render on a scanned pencil sketch using Canny edges at weight 1.0 for the full run. The result looks stiff and flat, and there are thin stray lines across the sky. What went wrong?"}],"options":[{"text":[{"text":"Canny is the wrong preprocessor for drawings and depth conditioning should be used instead."}],"correct":false,"tempting":[{"text":"Swaps the tool when the settings, not the choice of tool, produced both problems."}]},{"text":[{"text":"Canny keeps every edge it detects, including paper grain and scan artefacts, and holding the control at full weight for all the steps prevents the model from resolving the drawing into a rendered image; raising the thresholds and ending the control around 60% fixes both."}],"correct":true,"tempting":null},{"text":[{"text":"The sketch was scanned at too high a resolution, so far too many edges were detected."}],"correct":false,"tempting":[{"text":"Blames the input’s size for a problem caused by edge thresholds and control scheduling."}]},{"text":[{"text":"The run needs more denoising steps so the model has time to smooth the traced lines away."}],"correct":false,"tempting":[{"text":"Expects extra compute to overcome a constraint that is being re-applied at every one of those steps."}]}],"order":[2,0,3,1],"why":null},"prev":{"slug":"show-dont-describe","title":"Stop describing it and show it the picture","minutes":8},"next":{"slug":"the-img2img-ladder","title":"Three passes down the denoise ladder","minutes":8}}}