Why the hands were wrong

The two famous failures

For about eighteen months from mid-2022, you could identify a generated image in two places. The hands had six fingers, or four, or two thumbs, or a finger that merged into a cup. And any writing (a shop sign, a book cover, a T-shirt) was a plausible texture of letter-shapes that spelled nothing.

Both are gone, or mostly gone, in the models of 2024 onward. Understanding why they were there, and what fixed them, tells you more about how these systems learn than any success does.

Hands: a statistics problem

The model in this module learns what pictures look like from photographs. Think about how hands appear in photographs. They are small, usually a few per cent of the frame. They are often partly hidden, holding something, in a pocket, cut off by the edge. They take thousands of distinct poses, each of which looks different from every angle, and in most poses some fingers hide others. A face, by contrast, is large, usually shown front-on, and always has the same parts in the same arrangement.

So the model saw millions of clear faces and comparatively few clear hands, and the hands it saw did not agree with each other about how many fingers were visible. What it learned about hands was a texture: a skin-coloured region with finger-like protrusions, in an unspecified number. Asked to draw one at full size, it produced exactly that. The mechanism is the same one behind hallucination in the chatbot module (specific detail inside a familiar structure, with thin support) and the finger count was the invented specific.

The fix was not an idea about hands. It was more and better data: photographs where hands were large and clear, captions that described them, and larger models with more capacity for the rare case. A model that has seen enough clear hands learns that they usually have five fingers, in the same way it learned faces have two eyes. The improvement tracked the data, which is the honest way to say nobody taught it anatomy.

Letters: a reader problem

Text failed for a different reason, and the previous lesson set it up. The reader that converted the prompt into concept-positions saw the word “OPEN” as a single token and had no idea it was made of four letters in a particular order. The image model, meanwhile, had seen writing in photographs only as a texture (rows of letter-shaped marks) and learned to produce the texture. Neither half of the system knew what a letter was.

Two changes fixed it. Readers that see characters rather than whole words, so that the prompt “a sign saying OPEN” carries the letters O, P, E, N in order into the steering. And training captions that transcribed the visible text in images, so that the picture model learned that the texture of writing and the letters in the caption were the same thing. A model released in 2023 by a small company made legible text its main selling point; by 2024 the open models could do it too.

What is still wrong, and why

Long text still fails. A sign with three words works; a paragraph degrades, and the mechanism is the same as counting: the model holds a few letters as steered concepts and the rest as texture. Numbers of things (six people, exactly three windows) still fail for the reason in the first lesson of this module, that the model draws a texture of the thing rather than the things. Hands in complex poses, hands holding hands, hands playing instruments still go wrong more than faces do, because those remain rare in the data.

What this teaches about all of it

The hands were the honest signal of how these models work. They did not fail because the model lacked a rule about fingers; it has no rules. They failed because the statistics of the training photographs were thin exactly there. And they were fixed by thickening the statistics, not by cleverness. When you see any generative model fail at something specific, ask where in its training the support would have to come from, and whether it was ever there. It is the same question the chatbot module taught you to ask about a citation.