Predicting which second will break

A checklist you run before you spend

By this point in the course you know the mechanisms. This lesson turns them into a two-minute inspection you run on every still before it becomes a clip.

Open the frame at full size and look for nine things.

1. Contact points. Where a hand grips, a foot presses, a shoulder leans, an object rests. Contact is where a plausible frame becomes an impossible one, and the model has no constraint preventing a finger from passing through a strap. Count the contacts in the still. Each one is a place the clip can fail.

2. Thin structures. Railings, wires, chair legs, bicycle spokes, glasses frames, microphone stands. Thin means near the autoencoder’s resolution floor, so these are reinvented each frame and visibly flicker.

3. Text. Any letters at all. If they matter, they are coming out and going back on in the editor.

4. Reflections and speculars. Mirrors, windows, polished floors, car bodies, eyes. These require knowing where the light and surfaces are in three dimensions. The model approximates, and the approximation slides as the camera moves.

5. Background faces. A crowd is a set of faces, each too small to be resolved properly and each subject to the same identity drift as your lead. Background people churn. Blur them in the still, or accept it.

6. Repeating patterns. Tiles, brickwork, fences, striped shirts, keyboards. Regular grids are where the model’s failure to count becomes visible. Expect the count to change and the grid to warp.

7. Transparency. Glass, water, smoke, a veil. Transparent things require compositing two depths at once, and the model handles them as a texture rather than as a see-through surface. They tend to solidify or vanish.

8. Hair over a face. A partial occlusion that moves. Combines the two hardest things.

9. Anything you would have to count. Three birds, six candles, two rings, a hand with a specific number of fingers visible. Nothing is counting.

What to do with the list

For each item you find, pick one of four responses:

• Remove it from the still. Clone out the wire, blur the crowd, move the reflection out of frame.
• Reframe. A tighter shot that excludes the patterned floor. A different angle where the mirror is not visible.
• Reduce the move. Every item on the list gets worse with camera motion. A locked-off frame with a slight drift survives things a push-in does not.
• Shorten the shot. If the hand will fail at six seconds, the shot is four seconds long.

All four are cheaper than discovering the problem after generating.

Keep a log and find out how good you are

Write your prediction down before generating: which item you think will fail, and when. Then watch the clip and record what actually happened.

Twenty shots of this and you will have something valuable: a calibrated sense of your own model’s weaknesses, in your own subject matter. You will also discover where you are wrong. Most people over-predict text failures and under-predict background churn, because text is the famous problem and background churn is the quiet one.

A plain text file with four columns of your own devising is enough. This is the same habit an editor builds about a lens or a camera body, and it is what separates somebody who uses the tool from somebody who knows it.

The two-minute version

If you do nothing else, ask three questions of every still:

1. What is touching what?
2. What is thinner than a fingertip?
3. What would I have to count?

Those three catch most of it, and they take longer to read than to perform.

Honesty about the checklist

This list is a snapshot. Hands and short-range physics have improved measurably between model generations and will keep improving. Text in image models was a standing joke and largely is not any more.

So re-run the list yourself every few months on your own shots. Keep the mechanisms, which do not change, and treat the verdicts as provisional. A course that told you these were permanent would be lying, and a course that told you the next model fixes everything would also be lying: a specific real product is missing information, not missing capability, and that one does not move.

Today: run the nine-point check on the next still you make, write down which item you expect to fail, then generate and see whether you were right.