Reflections, shadows and things that must agree

The class of failure

Look closely at a generated image of a room with a mirror. The mirror contains a plausible reflection. It is frequently not a reflection of that room: the furniture is different, the angle is impossible, a figure appears who is not in the scene.

The same shape of failure recurs:

• Shadows that fall in different directions from different objects, or that are missing under one item and present under its neighbour.
• Water and glass that refract convincingly in each patch and inconsistently across the whole.
• Continuing lines — a railing, a cable, a road marking, the edge of a table — that go behind an object and re-emerge at a different height.
• Symmetry — a pair of earrings, two shoes, a jacket's buttons and buttonholes.
• Repeated instances of one thing, such as the same building at two scales, or a character's clothing changing between one side of the frame and the other.

Why this is one mechanism and not five

The denoising network operates over the latent grid with a limited receptive field per layer. Deeper layers see wider context, so the model does have some global reach — enough to keep a composition coherent. What it does not have is any constraint solver. Nothing checks that the reflection is consistent with the room, because consistency is not something the training objective ever asked for.

The objective was: given noisy input, predict the noise. A picture where the mirror shows the wrong room is, locally, exactly as easy to denoise as one where it shows the right room. Both are mirror-shaped regions containing mirror-like content. The training signal that would have distinguished them never existed.

This generalises into the most useful heuristic in the module:

These models are excellent at anything decided locally and unreliable at anything that must agree across the picture. Texture, material and light quality are local. Counting, spelling, reflection, symmetry and continuity are global.

Hands and text — the reused lesson opening this module — are the two most famous instances of the same rule. A hand must have a globally consistent number of fingers. A word must have globally consistent letters. Neither is a local texture problem.

What to do about it

Choose scenes that avoid the trap. This is the professional answer and it is not defeatist. Cinematographers avoid mirrors too, for their own reasons. If a reflection is not the point of the picture, do not put one in the frame.

Fix it in post. A wrong reflection is a two-minute job in any layered editor: select the mirror, paste a flipped copy of the actual scene, blur and reduce opacity. GIMP, Krita and Photopea all do this for nothing. This is far faster than re-rolling for a model to get lucky.

Inpaint the disagreeing region with the correct content visible as context. Masked regeneration sees the surrounding pixels, so a shadow inpainted with the light source in view often comes back correct.

Use a 3D pass for the geometry. For anything where physical consistency is the whole point — a product on a reflective surface, an architectural view — build the block-out in Blender, which is free and physically correct by construction, and use the render as a structural reference.

The claim to be sceptical of

Every generation of models is announced with improved "understanding of physics". Read this precisely. What improves is the statistical fidelity of local appearance and, in video, short-run plausibility of motion. What has not been demonstrated is a model that enforces a constraint.

The test that separates them is easy to run and worth running on any model that makes the claim: put a mirror or a still water surface in a scene with two distinctive objects, and check whether the reflection contains those objects in the right arrangement. If the answer is sometimes, the model is still doing statistics.

There is a version of this test for video too, and it is the one to reach for when a video model is described as a world simulator. Show it a scene where something is occluded and then revealed — a ball rolling behind a chair, a hand passing in front of a face. Local plausibility carries the shot until the object comes back out, and what comes back is often a different object, or nothing. Object permanence is a global constraint over time, which is the same category of thing as a reflection over space, and it fails for the same reason.