Addaly is in open beta. Things will change, and AI answers can be wrong — check anything that matters.

Making Things With AI

Images, video, voice and music — how they work, where they break, who owns them.

Lesson 32 of 849 min

Conditioning on structure: depth, edges and pose

The mechanism, briefly

A structural control takes an image-shaped map — an edge drawing, a depth field, a stick-figure skeleton — and feeds it into the denoising network alongside the text conditioning, at every step and at matching spatial positions.

The crucial difference from a text prompt is that the map is spatially aligned with the latent. Cell by cell, the network is told what should be there. There is no attention competition, no binding problem, no reliance on the caption corpus containing a word for the arrangement you want. This is why structural conditioning solves the problems the previous module said prompting cannot.

What a depth map pins, and what it leaves aloneFixed by the mapWhere every surface sits in spaceCamera position and perspectiveThe silhouette of each objectRelative scale between objectsThe composition you sketched in twenty secondsStill entirely the prompt's businessWhat the objects are made ofColour, light and time of dayThe medium — photograph, gouache, linocutHow many fingers are on the handWhether a sign spells anythingThe map is spatially aligned with the latent, cell by cell, so there is no attention competition andno reliance on the caption corpus having a word for your arrangement. That is why conditioning solveswhat prompting cannot, and why it fixes none of the failures that were never spatial.
What a depth map pins, and what it leavesaloneFixed by the mapWhere every surface sits in spaceCamera position and perspectiveThe silhouette of each objectRelative scale between objectsThe composition you sketched in twentysecondsStill entirely the prompt's businessWhat the objects are made ofColour, light and time of dayThe medium — photograph, gouache, linocutHow many fingers are on the handWhether a sign spells anythingThe map is spatially aligned with the latent, cellby cell, so there is no attention competition and noreliance on the caption corpus having a word foryour arrangement. That is why conditioning solveswhat prompting cannot, and why it fixes none of thefailures that were never spatial.

The best-known implementation is ControlNet, which trains a copy of part of the base model to accept the map. Others exist under different names — T2I-Adapter, and various built-in equivalents in hosted tools. The behaviour is close enough that the choice of map matters more than the choice of implementation.

Which map for which job

Canny edges. A hard black-and-white line drawing produced by an edge detector. Constrains outlines exactly. Use for: redrawing an existing image in a new style while keeping every contour, turning a line drawing into a rendered image, architectural work. Limitation: it is strict, so any noise in the edge detection becomes a line the model dutifully draws.

Depth. A greyscale map where brightness is distance. Constrains three-dimensional arrangement while leaving surface detail free. Use for: keeping a composition and camera while changing everything about the content. This is the most generally useful map and the best first choice.

Pose. A skeleton of joint positions. Constrains only the figure's arrangement, not its size, clothing or surroundings. Use for: putting a specific gesture or stance into an otherwise free image. Limitation: it carries no depth, so a figure can come back facing the wrong way, and hands are only a few points.

Soft edge and scribble. Loose line maps that constrain suggestion rather than outline. Use for: turning a rough sketch into an image without the model copying your bad lines.

Segmentation. A map of coloured regions, each colour meaning a category. Use for: laying out a scene by region — sky here, building there.

Normal maps and line art cover more specialised cases, and the same principle applies throughout: pick the map that constrains what you care about and nothing else.

The strength and timing settings

Two controls decide how hard the map is applied.

Conditioning strength. At 1.0 the map is followed closely; at 0.4 it is a suggestion. Very high strength on a noisy map produces artefacts, because the model is being forced to render detection noise.

Start and end steps. Applying the map only for the first 60% of steps fixes the composition and then releases the model to render freely. This is a genuinely useful trick: it gives you the layout without the map's stiffness, and it usually improves texture.

Where the maps come from

You do not need a photograph. All of these are useful sources:

  • A rough sketch, drawn in Krita, GIMP or on paper and photographed. Scribble and soft-edge maps are designed for exactly this.
  • A 3D block-out in Blender, which is free and gives you an exact depth map and exact camera control. For product shots and architecture this is the professional route and it is not difficult at the block level.
  • A photograph you took, run through a depth estimator. Most tools include one.
  • A pose you built in a free posing tool or extracted from any reference photograph.

The general point: you supply the geometry from something that is good at geometry, and the model supplies the surfaces, which is what it is good at.

The honest limits

Structural conditioning constrains where things are. It does not constrain what they are, and it does not fix any of the failures from the previous module that are not spatial. A depth map of a hand does not produce five fingers. A pose does not produce a consistent face. An edge map does not spell.

It also introduces its own failure: over-constraint. A model given a strong edge map of a photograph and asked for an oil painting will produce a photograph with paint texture, because every contour is pinned. If your restyling looks like a filter rather than a reinterpretation, lower the strength or switch from canny to depth. The fix is almost always to constrain less.

The one thing to keep

A control network injects a spatial map into the denoising loop at every step, fixing geometry while leaving content and style to the prompt, and each map type constrains a different thing.

Before you move on

You want to reinterpret a photograph as a woodcut print, keeping the composition but not the photographic detail. Which conditioning choice fits best?

Pick the one you would defend. Nobody sees your answer.

No ads. No data sale. No public scores on people. Ever.

© 2026 Addaly