{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"memorisation-and-regurgitation","title":"Memorisation, regurgitation, and the dispute that hangs on them","url":"https://addaly.com/learn/how-llms-work/memorisation-and-regurgitation","position":76,"of":78,"minutes":10,"answer":"A model stores no copies, yet text duplicated many times in the corpus gets a path through the weights that a prefix can retrieve verbatim: memorisation is driven by duplication, measurable by extraction tests.","keyIdea":"A model stores no copies, yet text duplicated many times in the corpus gets a path through the weights that a prefix can retrieve verbatim: memorisation is driven by duplication, measurable by extraction tests, and distinct from the stylistic imitation it is usually confused with."},"body":{"format":"markdown","text":"## Two claims that are both true\n\nModule one showed you a model file: numbers, a configuration, a tokenizer, and no copy of anything. Module five noted that text seen many times in training can sometimes be reproduced. Both are true, and the space between them is where measurement, mitigation and a set of unresolved legal disputes all live.\n\n## What memorisation is, mechanically\n\nTraining reduces loss on the corpus. For a passage that appears once, the cheapest way to reduce loss is to learn the general patterns it shares with everything else. For a passage that appears hundreds of times, the cheapest way is to **store it**: to build a path through the weights, of the key-and-lookup kind the previous lessons found, long enough that a prefix of the passage retrieves the rest verbatim.\n\nSo the driver is **duplication**. A sequence seen once is almost never extractable. Seen dozens of times, sometimes. Seen hundreds, often. That is why deduplication sits in the data pipeline of module five, and it is why the passages that come back tend to be the ones the web copied most: licence texts, famous poems, popular novels' opening pages, widely mirrored code, and, worryingly, a person's contact details from a page that was scraped two hundred times.\n\nTwo other regularities: larger models memorise more at the same duplication rate, and longer prompts extract more, because a longer prefix selects the path more precisely.\n\n## The measurements\n\nThe standard test takes a passage known to be in the training data, gives the model its first fifty tokens, and checks whether the next fifty come back exactly. On open models with published corpora, around one per cent of sampled training sequences are extractable this way, rising steeply with how many times they appeared and with model size.\n\nIn 2023 a stranger attack was published. Ask a chat model to repeat a single word forever. After a while it stops, and what follows is training text: fragments of web pages, code, and in the documented cases names, email addresses and phone numbers. The mechanism is instructive: the repetition pushes the model out of the region where its chat training holds and into base-model behaviour, where memorised sequences surface. The provider patched the specific trigger. The content it revealed was still in the weights, which is the point of the unlearning lesson that follows.\n\nThree levels need separating. **Verbatim** reproduction is rare and measurable. **Near-verbatim** (a passage with small changes) is more common. **Stylistic** imitation is universal and is not memorisation at all; a model writing in the manner of an author has learned patterns, not stored text, and conflating the three is the commonest error in public discussion of this subject.\n\n## Mitigations, and what each does\n\n- **Deduplicate the training data.** The largest single effect; it removes the driver.\n- **Filter personal data before training.** Imperfect (module six's uncertainty about what a filter catches applies) and far better than nothing.\n- **Differential privacy in training** adds noise to every update so that no single example can be memorised, with a formal guarantee. It costs model quality measurably and is rarely used for large-scale pretraining, though it is used for fine-tuning on sensitive data.\n- **Output-side matching.** Compare generated text against an index of protected sources and block matches. Some providers do this for code. It catches verbatim, misses near-verbatim.\n- **Preference training** reduces regurgitation in ordinary use without removing the content, as the repetition attack showed.\n\n## The dispute, in shape only\n\nSeveral legal questions hang on the mechanisms above, and they are unresolved and differ by country. Is training on copyrighted text an infringement? Is a verbatim output one? Is the model itself a derivative work? Who is liable: the trainer, the deployer, the user?\n\nThe shape, without resolution: in the United States the question runs through fair use, with litigation ongoing and early rulings in 2025 pointing different ways on different facts, notably distinguishing how the training data was acquired from what the training did with it. The European Union has a text-and-data-mining exception that rightsholders can opt out of, and transparency duties on training-data summaries. Japan has a broad exception permitting training on copyrighted works, with limits. The United Kingdom has a narrow exception for non-commercial research and has been consulting on a wider one. India has no provision written for this, a fair-dealing doctrine narrower than American fair use, and cases pending.\n\nThis lesson gives no legal advice and takes no side. What the mechanism contributes is a distinction the arguments often blur: \"the model learned from this text\" and \"the model can reproduce this text\" are different claims, the second is measurable and reducible, and which one is being made changes what any rule would have to say.\n\n## What you can do\n\nIf you train or fine-tune: deduplicate, strip personal data, and keep provenance (which sources, under which licence) because you will be asked. If you fine-tune a shared model on one customer's documents, assume another customer can extract some of them, and use the per-customer adapters that module five described precisely for this. If you deploy: run the fifty-token extraction test on your own fine-tuning data. The test is a script, the deduplication tools are free, and the published memorisation measurements on open models are reproducible on a laptop."},"check":{"question":"A model reproduces a 200-word passage from a popular novel verbatim but cannot reproduce a passage from an obscure one, though both books were in the training data. What is the mechanism?","options":[{"text":"Popular novels were included again in the fine-tuning stage, which is where verbatim recall is learned.","correct":false,"tempting":"Fine-tuning data is small and behaviour-focused; verbatim recall of long passages comes from pretraining exposure, and the obscure book was in that stage too."},{"text":"The model keeps complete copies of widely read works in a dedicated part of its weights, and the obscure book did not qualify.","correct":false,"tempting":"Module one's point stands: there are no copies and no dedicated part; what differs is how strongly a retrieval path was reinforced."},{"text":"The obscure passage tokenises into rarer tokens that the model handles less accurately, so its recall breaks down mid-sentence.","correct":false,"tempting":"Tokenisation affects many things, but both books are ordinary English prose with ordinary tokens; the difference is in how often the sequence itself was seen."},{"text":"The popular passage appeared many times across the corpus, and duplication drives verbatim memorisation.","correct":true}]},"prev":{"slug":"is-the-reasoning-faithful","title":"Is the written reasoning what the model did?","url":"https://addaly.com/learn/how-llms-work/is-the-reasoning-faithful"},"next":{"slug":"unlearning","title":"Unlearning: why removing something is harder than adding it","url":"https://addaly.com/learn/how-llms-work/unlearning"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Memorisation, regurgitation, and the dispute that hangs on them', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/memorisation-and-regurgitation (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Memorisation, regurgitation, and the dispute that hangs on them","courseTitle":"How a Language Model Actually Works","module":{"index":8,"slug":"looking-inside","title":"Looking inside: what the weights actually hold"},"answer":null,"keyIdea":"A model stores no copies, yet text duplicated many times in the corpus gets a path through the weights that a prefix can retrieve verbatim: memorisation is driven by duplication, measurable by extraction tests, and distinct from the stylistic imitation it is usually confused with.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-claims-that-are-both-true","runs":[{"text":"Two claims that are both true"}]},{"t":"p","runs":[{"text":"Module one showed you a model file: numbers, a configuration, a tokenizer, and no copy of anything. Module five noted that text seen many times in training can sometimes be reproduced. Both are true, and the space between them is where measurement, mitigation and a set of unresolved legal disputes all live."}]},{"t":"h2","id":"what-memorisation-is-mechanically","runs":[{"text":"What memorisation is, mechanically"}]},{"t":"p","runs":[{"text":"Training reduces loss on the corpus. For a passage that appears once, the cheapest way to reduce loss is to learn the general patterns it shares with everything else. For a passage that appears hundreds of times, the cheapest way is to "},{"text":"store it","b":true},{"text":": to build a path through the weights, of the key-and-lookup kind the previous lessons found, long enough that a prefix of the passage retrieves the rest verbatim."}]},{"t":"p","runs":[{"text":"So the driver is "},{"text":"duplication","b":true},{"text":". A sequence seen once is almost never extractable. Seen dozens of times, sometimes. Seen hundreds, often. That is why deduplication sits in the data pipeline of module five, and it is why the passages that come back tend to be the ones the web copied most: licence texts, famous poems, popular novels’ opening pages, widely mirrored code, and, worryingly, a person’s contact details from a page that was scraped two hundred times."}]},{"t":"p","runs":[{"text":"Two other regularities: larger models memorise more at the same duplication rate, and longer prompts extract more, because a longer prefix selects the path more precisely."}]},{"t":"h2","id":"the-measurements","runs":[{"text":"The measurements"}]},{"t":"p","runs":[{"text":"The standard test takes a passage known to be in the training data, gives the model its first fifty tokens, and checks whether the next fifty come back exactly. On open models with published corpora, around one per cent of sampled training sequences are extractable this way, rising steeply with how many times they appeared and with model size."}]},{"t":"p","runs":[{"text":"In 2023 a stranger attack was published. Ask a chat model to repeat a single word forever. After a while it stops, and what follows is training text: fragments of web pages, code, and in the documented cases names, email addresses and phone numbers. The mechanism is instructive: the repetition pushes the model out of the region where its chat training holds and into base-model behaviour, where memorised sequences surface. The provider patched the specific trigger. The content it revealed was still in the weights, which is the point of the unlearning lesson that follows."}]},{"t":"p","runs":[{"text":"Three levels need separating. "},{"text":"Verbatim","b":true},{"text":" reproduction is rare and measurable. "},{"text":"Near-verbatim","b":true},{"text":" (a passage with small changes) is more common. "},{"text":"Stylistic","b":true},{"text":" imitation is universal and is not memorisation at all; a model writing in the manner of an author has learned patterns, not stored text, and conflating the three is the commonest error in public discussion of this subject."}]},{"t":"h2","id":"mitigations-and-what-each-does","runs":[{"text":"Mitigations, and what each does"}]},{"t":"ul","items":[[{"text":"Deduplicate the training data.","b":true},{"text":" The largest single effect; it removes the driver."}],[{"text":"Filter personal data before training.","b":true},{"text":" Imperfect (module six’s uncertainty about what a filter catches applies) and far better than nothing."}],[{"text":"Differential privacy in training","b":true},{"text":" adds noise to every update so that no single example can be memorised, with a formal guarantee. It costs model quality measurably and is rarely used for large-scale pretraining, though it is used for fine-tuning on sensitive data."}],[{"text":"Output-side matching.","b":true},{"text":" Compare generated text against an index of protected sources and block matches. Some providers do this for code. It catches verbatim, misses near-verbatim."}],[{"text":"Preference training","b":true},{"text":" reduces regurgitation in ordinary use without removing the content, as the repetition attack showed."}]]},{"t":"h2","id":"the-dispute-in-shape-only","runs":[{"text":"The dispute, in shape only"}]},{"t":"p","runs":[{"text":"Several legal questions hang on the mechanisms above, and they are unresolved and differ by country. Is training on copyrighted text an infringement? Is a verbatim output one? Is the model itself a derivative work? Who is liable: the trainer, the deployer, the user?"}]},{"t":"p","runs":[{"text":"The shape, without resolution: in the United States the question runs through fair use, with litigation ongoing and early rulings in 2025 pointing different ways on different facts, notably distinguishing how the training data was acquired from what the training did with it. The European Union has a text-and-data-mining exception that rightsholders can opt out of, and transparency duties on training-data summaries. Japan has a broad exception permitting training on copyrighted works, with limits. The United Kingdom has a narrow exception for non-commercial research and has been consulting on a wider one. India has no provision written for this, a fair-dealing doctrine narrower than American fair use, and cases pending."}]},{"t":"p","runs":[{"text":"This lesson gives no legal advice and takes no side. What the mechanism contributes is a distinction the arguments often blur: “the model learned from this text” and “the model can reproduce this text” are different claims, the second is measurable and reducible, and which one is being made changes what any rule would have to say."}]},{"t":"h2","id":"what-you-can-do","runs":[{"text":"What you can do"}]},{"t":"p","runs":[{"text":"If you train or fine-tune: deduplicate, strip personal data, and keep provenance (which sources, under which licence) because you will be asked. If you fine-tune a shared model on one customer’s documents, assume another customer can extract some of them, and use the per-customer adapters that module five described precisely for this. If you deploy: run the fifty-token extraction test on your own fine-tuning data. The test is a script, the deduplication tools are free, and the published memorisation measurements on open models are reproducible on a laptop."}]}],"check":{"question":[{"text":"A model reproduces a 200-word passage from a popular novel verbatim but cannot reproduce a passage from an obscure one, though both books were in the training data. What is the mechanism?"}],"options":[{"text":[{"text":"Popular novels were included again in the fine-tuning stage, which is where verbatim recall is learned."}],"correct":false,"tempting":[{"text":"Fine-tuning data is small and behaviour-focused; verbatim recall of long passages comes from pretraining exposure, and the obscure book was in that stage too."}]},{"text":[{"text":"The model keeps complete copies of widely read works in a dedicated part of its weights, and the obscure book did not qualify."}],"correct":false,"tempting":[{"text":"Module one’s point stands: there are no copies and no dedicated part; what differs is how strongly a retrieval path was reinforced."}]},{"text":[{"text":"The obscure passage tokenises into rarer tokens that the model handles less accurately, so its recall breaks down mid-sentence."}],"correct":false,"tempting":[{"text":"Tokenisation affects many things, but both books are ordinary English prose with ordinary tokens; the difference is in how often the sequence itself was seen."}]},{"text":[{"text":"The popular passage appeared many times across the corpus, and duplication drives verbatim memorisation."}],"correct":true,"tempting":null}],"order":[0,2,3,1],"why":null},"prev":{"slug":"is-the-reasoning-faithful","title":"Is the written reasoning what the model did?","minutes":9},"next":{"slug":"unlearning","title":"Unlearning: why removing something is harder than adding it","minutes":9}}}