Two famous failures
In 2022, hands. Six fingers, thumbs branching off thumbs, a knuckle where a nail should be. And shop signs reading RESTAVRANT or BAKREY — letter-shaped marks arranged with total confidence into nothing.
Both are largely fixed. Understanding *why* they broke is worth much more than the fix, because it tells you what else will break.
Hands
A hand is small in the frame, has roughly 27 degrees of freedom, appears in thousands of configurations, and is constantly blurred, cropped or half-hidden in training photos. And no caption ever says "with five fingers", because no human writes that.
So what could the model learn? Finger-ish texture. Skin, knuckle, nail, the taper of a fingertip. What it could not learn from that training objective is a *count*, because a count is never locally visible. Standing at any single patch of the image, four fingers and six fingers look equally plausible. Local plausibility is all the objective ever rewarded.
Text
Spelling is worse, because it is an exact, ordered, discrete constraint stretched across a distance. The older text encoders made it close to impossible: they chop your prompt into subword tokens and hand the image model a semantic vector. There is no character-level information in there. The model was asked to render BAKERY while genuinely not being told which letters that word contains. Letter-shaped marks were the honest best guess.
What actually fixed them
Not a clever trick. Four unglamorous things:
- Bigger, character-aware text encoders that can see the actual string.
- Higher working resolution, so a hand occupies enough of the compressed space to be represented at all.
- Recaptioning the training data with vision models, so captions describe what is in the picture rather than whatever alt-text a website happened to have.
- Scale.
The pattern worth keeping
These models are extremely good at locally plausible texture and weak wherever correctness is a global, discrete, exact constraint — something you can only verify by taking in the whole image and counting or comparing.
That predicts failures you have not hit yet:
- Exact quantities. Nine coins, three windows, a hand of five cards.
- Reflections and shadows that must agree with the scene rather than merely look like reflections.
- Clock faces, calendars, dice, sheet music, chessboards, wiring diagrams.
- Anything repeated identically: a logo appearing twice in one image, a tiled pattern, the same character in two separate pictures.
- Maps, and writing in scripts thin on the ground in training data. Devanagari, Arabic and Thai still fail in models that spell English perfectly, for exactly the reason English used to fail.
What to do about it
Stop trying to prompt your way through a global constraint.
- Put real text on afterwards, in a real font, in a design tool. Nobody in production generates a poster headline.
- Generate the countable thing separately, or crop so it sits out of frame.
- Use masked editing and structural conditioning for the parts that must be exact. That is the next lesson.
- For a character who must recur, use the model's reference or consistency features rather than describing the person again and hoping.
One more use for this
The same reasoning applies to spotting synthetic images. As models improve, the tells stop being fingers and become global consistency: a reflection that does not match the room, shadows at two different angles, earrings that change between two shots of the same person, background text that reads fine from a distance and dissolves up close.
Fingers were never the point. Counting was.
Before you move on