What the model does and does not retain

The size argument, and why it is not the whole answer

The most-repeated defence of these systems is arithmetic. Stable Diffusion 1.5 has roughly 860 million parameters, about 2 gigabytes on disk in half precision. It was trained on billions of images. If it stored them, it would be storing each in a fraction of a byte, which is impossible. Therefore it cannot contain the training images.

The arithmetic is right and the conclusion is too strong. Average compression across the whole set is not the same as compression of any particular item. A model can be an extreme lossy summary of nearly everything and still hold a handful of specific images almost exactly — precisely the ones it saw thousands of times.

What the measurements say

This has been tested rather than argued. The best-known study, by Carlini and colleagues in 2023, took Stable Diffusion, selected the training captions with the most duplicated images, and generated a very large number of samples — on the order of 175 million — checking each against the training set.

They recovered roughly a hundred near-exact copies. That is a small number against 175 million generations, and it is not zero. Every recovered image shared one property: it appeared many times in the training data, typically hundreds of times, usually because it was a stock photo, a film poster, a book cover or a widely reposted meme.

The mechanism is straightforward once you have the earlier lessons. Training reduces prediction error. An image seen once contributes a nudge that is averaged away among millions of others. An image seen a thousand times, with the same caption, becomes a reliable low-error target — the model can lower its loss by learning that specific mapping. Memorisation is not a bug in the architecture; it is what optimisation does with duplicates.

The consequences you should carry

Deduplication is the fix, and it is now standard. Later training runs remove near-duplicate images before training, which drops the memorisation rate substantially. When a lab says its dataset was deduplicated, this is the risk it is talking about.

Named artists and named works behave differently from styles. A prompt naming a very famous single painting has a real chance of producing something close to it, because that painting appears in the training data thousands of times. A prompt naming a living illustrator with a modest online presence usually produces a loose imitation of their style rather than a copy of any one work — which raises a different question, since style is not protected by copyright in most jurisdictions while a specific reproduction is.

"The model cannot copy" is not a safe thing to tell a client. The honest statement is that copying is rare, is concentrated in heavily duplicated images, and is your responsibility to check when the output will be published. The check is cheap: reverse image search the result. Google Lens, TinEye and Yandex are free, take fifteen seconds, and will catch the poster or stock photo that came back nearly intact.

Faces, and the sharper version of the problem

The same duplication effect applies to people. Individuals who appear in the training scrape thousands of times — actors, politicians, anyone with a heavy image presence — can be produced recognisably by name. Individuals who appear a few times generally cannot. This is why the practical risk of a model generating an identifiable private person by accident is low, and why the risk of generating a public figure in a fabricated situation is high enough that most services block it by name.

The limitation nobody has solved: you cannot inspect a model to find out what it memorised. There is no index to query. The only method available is the one the researchers used — generate a great deal and compare against the training set — and that requires access to the training set, which most commercial models do not publish. So for a closed model, nobody outside the lab can tell you what it holds. That is worth saying out loud when someone offers you an indemnity based on a claim about training data you cannot verify.

One more distinction is worth keeping straight, because the two get argued as though they were one thing. Reproducing a specific image is a copyright question with a fairly clear shape in most countries. Reproducing a style is not — style as such is generally outside copyright, which is why a great deal of the anger about these systems does not map onto a cause of action, and why several artists' claims have been reframed around trademark, passing off or contract instead. That is covered properly in the final module. What the memorisation evidence settles is only the narrow question: near-exact copies are possible, they are rare, and they cluster on duplicates.