{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"where-the-text-came-from","title":"Where the text came from, and whether it is running out","url":"https://addaly.com/learn/ai-actually-explained/where-the-text-came-from","position":51,"of":92,"minutes":10,"answer":"A model's voice is the filtered residue of the web, and the stock of public human text is finite enough that the labs expect to have used it within a few years.","keyIdea":"A model's voice is the filtered residue of the web, and the stock of public human text is finite enough that the labs expect to have used it within a few years."},"body":{"format":"markdown","text":"## The pile\n\nThe chatbot module told you the model learned from \"the pile\". This lesson is about what was in it, because the contents explain the voice, the gaps, and several lawsuits.\n\nThe base of nearly every large model is a crawl of the public web. The largest is run by a non-profit that has been saving copies of web pages since 2008, and it now holds petabytes across hundreds of billions of pages. It is free to anyone, and it is also, by language, about forty-five per cent English. Hindi is a fraction of one per cent, which is the mechanism behind the lesson on being weaker in your language.\n\nOn top of the crawl sit the sources a lab thinks are higher quality: the whole of Wikipedia, digitised books, academic papers, public code from GitHub, question-and-answer sites, forum threads, and transcripts of spoken video. Each has a story. A collection of nearly two hundred thousand books, assembled from a pirate library, was used in early open models and is now the central exhibit in several copyright cases. Forum and question-and-answer sites that were once crawled for free now sell access; one deal reported in 2024 was sixty million dollars a year.\n\n## The scale\n\nGPT-3 was trained on about 300 billion tokens. Meta's 2024 models were trained on fifteen trillion: fifty times more in four years. To place that: the whole of English Wikipedia is on the order of five billion tokens, which is about three hundredths of one per cent of what those models read. The encyclopaedia everyone imagines as the model's main source is a rounding error in the mix. It matters more than its size suggests because labs give it extra weight, but it is not the pile.\n\n## The filter is the voice\n\nRaw crawl is mostly unusable: navigation menus, boilerplate, spam, duplicated pages, machine-generated filler. Before training, a lab removes duplicates, strips the obvious junk, and then runs a classifier that keeps text resembling the sources it trusts (reference writing, published books, well-edited articles) and drops the rest.\n\nThe consequence is that the surviving text has a register. It reads like an encyclopaedia entry crossed with a helpful forum reply, because those are what the filter kept. When a model's answer sounds like a Wikipedia paragraph with a friendly opening, you are hearing the filter, not a personality. The second training adjusts the manners; the register underneath was set by what was kept.\n\n## Running out\n\nHere is the number that shapes the labs' planning. A research group that tracks the industry estimated in 2024 that the total stock of public human-written text of usable quality is on the order of three hundred trillion tokens, and that at current growth in training-set size it will be fully used somewhere between 2026 and 2032. Text is not being written fast enough to keep up. The scaling laws say more text keeps helping; the world does not have it.\n\nFour responses are under way, and the honest summary is that none is proven.\n\n**Licensing.** Paying publishers, news agencies and forums for text that was not on the open web, or for the right to keep using what was. The deals are in the tens to hundreds of millions each.\n\n**Other kinds of data.** Images, audio and video are far more abundant than text, and a model that reads them all is not limited by the text stock alone.\n\n**Synthetic text.** Having a strong model write textbooks, exercises and dialogues for a smaller one to learn from. Several small models of 2023 and 2024 were trained heavily this way and performed above their size. Whether it works at the frontier (whether a model can teach a successor more than it knows) is open.\n\n**Reading the same text more than once.** Which helps, with diminishing returns after a few passes.\n\n## The collapse worry\n\nA 2024 paper in *Nature* trained models repeatedly on their own output and watched them degrade: each generation lost the rare cases and converged on the common ones, until the text became repetitive nonsense. The concern is that the web is now filling with model output, so future crawls will be partly a model's own reflection.\n\nThe experiment was deliberately extreme, with each generation trained only on the last. Real training sets mix fresh human text with the rest, and subsequent work suggests the mix is stable if the human share is kept. But nobody yet knows the proportion of the 2025 web that is generated, and nobody can filter it reliably, because the detectors do not work. This is unsettled, and it is one of the few places where a mechanism you have already learned (the loss of the tails) points at a risk to the models themselves."},"check":{"question":"Why does a chatbot's answer so often read like an encyclopaedia entry with a friendly opening?","options":[{"text":"Because the developers write a formal, encyclopaedic style into the model's hidden instructions, and the model follows them.","correct":false,"tempting":"Locates the register in the instructions; those adjust manners, and the underlying register comes from the training text."},{"text":"Because the quality filters kept reference-style text and forum replies, and that mix is what it learned to continue.","correct":true},{"text":"Because Wikipedia makes up most of the training data.","correct":false,"tempting":"Overestimates the encyclopaedia; it is a few hundredths of one per cent of a modern training set."},{"text":"Because English as a language has a naturally formal register.","correct":false,"tempting":"Blames the language; the same filter applied to any language would produce the same effect."}]},"prev":{"slug":"what-it-cost-to-build","title":"What it cost to build one","url":"https://addaly.com/learn/ai-actually-explained/what-it-cost-to-build"},"next":{"slug":"open-weights-and-closed","title":"Open weights, closed weights, and what \"open\" is hiding","url":"https://addaly.com/learn/ai-actually-explained/open-weights-and-closed"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Where the text came from, and whether it is running out', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/where-the-text-came-from (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Where the text came from, and whether it is running out","courseTitle":"AI, Actually Explained","module":{"index":5,"slug":"how-it-got-here","title":"How it got here, and what changed in 2022"},"answer":null,"keyIdea":"A model’s voice is the filtered residue of the web, and the stock of public human text is finite enough that the labs expect to have used it within a few years.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-pile","runs":[{"text":"The pile"}]},{"t":"p","runs":[{"text":"The chatbot module told you the model learned from “the pile”. This lesson is about what was in it, because the contents explain the voice, the gaps, and several lawsuits."}]},{"t":"p","runs":[{"text":"The base of nearly every large model is a crawl of the public web. The largest is run by a non-profit that has been saving copies of web pages since 2008, and it now holds petabytes across hundreds of billions of pages. It is free to anyone, and it is also, by language, about forty-five per cent English. Hindi is a fraction of one per cent, which is the mechanism behind the lesson on being weaker in your language."}]},{"t":"p","runs":[{"text":"On top of the crawl sit the sources a lab thinks are higher quality: the whole of Wikipedia, digitised books, academic papers, public code from GitHub, question-and-answer sites, forum threads, and transcripts of spoken video. Each has a story. A collection of nearly two hundred thousand books, assembled from a pirate library, was used in early open models and is now the central exhibit in several copyright cases. Forum and question-and-answer sites that were once crawled for free now sell access; one deal reported in 2024 was sixty million dollars a year."}]},{"t":"h2","id":"the-scale","runs":[{"text":"The scale"}]},{"t":"p","runs":[{"text":"GPT-3 was trained on about 300 billion tokens. Meta’s 2024 models were trained on fifteen trillion: fifty times more in four years. To place that: the whole of English Wikipedia is on the order of five billion tokens, which is about three hundredths of one per cent of what those models read. The encyclopaedia everyone imagines as the model’s main source is a rounding error in the mix. It matters more than its size suggests because labs give it extra weight, but it is not the pile."}]},{"t":"h2","id":"the-filter-is-the-voice","runs":[{"text":"The filter is the voice"}]},{"t":"p","runs":[{"text":"Raw crawl is mostly unusable: navigation menus, boilerplate, spam, duplicated pages, machine-generated filler. Before training, a lab removes duplicates, strips the obvious junk, and then runs a classifier that keeps text resembling the sources it trusts (reference writing, published books, well-edited articles) and drops the rest."}]},{"t":"p","runs":[{"text":"The consequence is that the surviving text has a register. It reads like an encyclopaedia entry crossed with a helpful forum reply, because those are what the filter kept. When a model’s answer sounds like a Wikipedia paragraph with a friendly opening, you are hearing the filter, not a personality. The second training adjusts the manners; the register underneath was set by what was kept."}]},{"t":"h2","id":"running-out","runs":[{"text":"Running out"}]},{"t":"p","runs":[{"text":"Here is the number that shapes the labs’ planning. A research group that tracks the industry estimated in 2024 that the total stock of public human-written text of usable quality is on the order of three hundred trillion tokens, and that at current growth in training-set size it will be fully used somewhere between 2026 and 2032. Text is not being written fast enough to keep up. The scaling laws say more text keeps helping; the world does not have it."}]},{"t":"p","runs":[{"text":"Four responses are under way, and the honest summary is that none is proven."}]},{"t":"p","runs":[{"text":"Licensing.","b":true},{"text":" Paying publishers, news agencies and forums for text that was not on the open web, or for the right to keep using what was. The deals are in the tens to hundreds of millions each."}]},{"t":"p","runs":[{"text":"Other kinds of data.","b":true},{"text":" Images, audio and video are far more abundant than text, and a model that reads them all is not limited by the text stock alone."}]},{"t":"p","runs":[{"text":"Synthetic text.","b":true},{"text":" Having a strong model write textbooks, exercises and dialogues for a smaller one to learn from. Several small models of 2023 and 2024 were trained heavily this way and performed above their size. Whether it works at the frontier (whether a model can teach a successor more than it knows) is open."}]},{"t":"p","runs":[{"text":"Reading the same text more than once.","b":true},{"text":" Which helps, with diminishing returns after a few passes."}]},{"t":"h2","id":"the-collapse-worry","runs":[{"text":"The collapse worry"}]},{"t":"p","runs":[{"text":"A 2024 paper in "},{"text":"Nature","i":true},{"text":" trained models repeatedly on their own output and watched them degrade: each generation lost the rare cases and converged on the common ones, until the text became repetitive nonsense. The concern is that the web is now filling with model output, so future crawls will be partly a model’s own reflection."}]},{"t":"p","runs":[{"text":"The experiment was deliberately extreme, with each generation trained only on the last. Real training sets mix fresh human text with the rest, and subsequent work suggests the mix is stable if the human share is kept. But nobody yet knows the proportion of the 2025 web that is generated, and nobody can filter it reliably, because the detectors do not work. This is unsettled, and it is one of the few places where a mechanism you have already learned (the loss of the tails) points at a risk to the models themselves."}]}],"check":{"question":[{"text":"Why does a chatbot’s answer so often read like an encyclopaedia entry with a friendly opening?"}],"options":[{"text":[{"text":"Because the developers write a formal, encyclopaedic style into the model’s hidden instructions, and the model follows them."}],"correct":false,"tempting":[{"text":"Locates the register in the instructions; those adjust manners, and the underlying register comes from the training text."}]},{"text":[{"text":"Because the quality filters kept reference-style text and forum replies, and that mix is what it learned to continue."}],"correct":true,"tempting":null},{"text":[{"text":"Because Wikipedia makes up most of the training data."}],"correct":false,"tempting":[{"text":"Overestimates the encyclopaedia; it is a few hundredths of one per cent of a modern training set."}]},{"text":[{"text":"Because English as a language has a naturally formal register."}],"correct":false,"tempting":[{"text":"Blames the language; the same filter applied to any language would produce the same effect."}]}],"order":[0,1,2,3],"why":null},"prev":{"slug":"what-it-cost-to-build","title":"What it cost to build one","minutes":9},"next":{"slug":"open-weights-and-closed","title":"Open weights, closed weights, and what “open” is hiding","minutes":9}}}