{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"what-it-read","title":"What was in the pile","url":"https://addaly.com/learn/ai-actually-explained/what-it-read","position":27,"of":92,"minutes":9,"answer":"A model is a compression of what was written down, in public, in that language, before the cut-off, and its fluency stays uniform even where its knowledge is thin.","keyIdea":"A model is a compression of what was written down, in public, in that language, before the cut-off, and its fluency stays uniform even where its knowledge is thin."},"body":{"format":"markdown","text":"## The pile decides the model\n\nA model is a compression of what it was trained on. So the honest way to predict what it will be good at is to ask what was in the pile.\n\nFor a large language model the pile is text on the order of ten to fifteen trillion tokens: a quantity nobody could read, roughly equivalent to tens of millions of books. It is assembled from a handful of recognisable sources:\n\n- **Web crawl.** The bulk of it. Common Crawl, a free public archive of scraped web pages, is the starting point for most open efforts, filtered heavily.\n- **Wikipedia**, in many languages, weighted far above its size because it is clean and factual.\n- **Books**, from public-domain collections and, contentiously, from other sources.\n- **Code**, largely from public repositories on GitHub. Training on code appears to improve reasoning on non-code tasks, which was not the plan.\n- **Forums and question sites** (Reddit, Stack Exchange) which teach conversational structure and the shape of an answer.\n- **Scientific papers**, from arXiv and PubMed.\n- Increasingly, **synthetic text** generated by other models.\n\n## What is not in the pile\n\nMore useful than the list is its shadow.\n\n**Anything not written down.** Craft knowledge that passes by demonstration (how a fabric should feel, how an engine sounds when the bearing is going) is thinly represented, because people who know it write it down rarely.\n\n**Anything behind a wall.** Textbooks in copyright, most journalism after a paywall, court records that are not digitised, internal company documents, most of WhatsApp and every private conversation.\n\n**Anything in a language with a small web presence.** This is the big one for most of the world. English is perhaps five per cent of humanity and a large majority of the training text. A language with a hundred million speakers and a thin internet is a language the model is weak in, and it will not say so.\n\n**Anything from before the internet that nobody scanned.** Local history, regional legal practice, oral tradition.\n\n**Anything after the cut-off**, which has its own lesson.\n\n## Filtering, and what filtering does\n\nRaw crawl is largely unusable: boilerplate, navigation, spam, duplicates, machine-translated sludge. So it is filtered: deduplicated, scored by quality classifiers, and screened for certain categories of harmful content.\n\nDeduplication matters more than it sounds. Training on near-identical copies of the same page makes memorisation likelier and generalisation worse, so labs spend serious effort on it.\n\nBut a quality classifier is itself a model with a view about what good text looks like, and that view was formed from examples somebody chose. Filtering is not neutral, and there is documented evidence that some filters disproportionately remove text written in non-standard dialects. Cleaning a corpus is an editorial act.\n\n## What is knowable, and what is now secret\n\nThis is worth being blunt about. Until around 2022, papers described training data in reasonable detail. Now the leading commercial labs do not disclose theirs, for competitive and legal reasons. When someone tells you what GPT-class models were trained on, they are inferring.\n\nOpen alternatives exist and are valuable if you want to see inside one:\n\n- **OLMo**, from the Allen Institute for AI, ships with its full training corpus, `Dolma`, published and downloadable.\n- **The Pile** and **RedPajama** are open, documented datasets you can inspect.\n- **Pythia** is a set of models released with their data and intermediate checkpoints specifically so researchers can study what training does.\n\nIf you ever want to move from believing things about training data to checking them, those are the doors.\n\n## The one habit to take away\n\nBefore trusting a model on a subject, ask: **was this written down, in quantity, in public, in this language, before the cut-off?**\n\nFour yeses and the model is probably solid. A no on any of them and you should expect confident output of unknown quality, because the model's fluency is uniform, and its knowledge is not.\n\nThat asymmetry, between even fluency and uneven knowledge, is the single most dangerous property of these systems, and it comes directly from the pile."},"check":{"question":"A user asks a model detailed questions about tenancy customs in a small district, and gets fluent, specific, confident answers. What is the most reasonable stance?","options":[{"text":"The fluency indicates the model has relevant training material, since it would hedge if it did not.","correct":false,"tempting":"Assumes fluency tracks knowledge; the model's fluency is uniform and does not degrade where its material is thin."},{"text":"Local legal detail is rarely written up in quantity online, so specificity here is a warning rather than a reassurance, and each claim needs a source.","correct":true},{"text":"The answers are likely drawn from national law, which is well documented, so they will be broadly right.","correct":false,"tempting":"Guesses at a plausible substitution the model might make, when what the model produces is a generated blend rather than a documented fallback."},{"text":"Asking the model to state its confidence would separate the well-supported claims from the invented ones.","correct":false,"tempting":"Treats stated confidence as a readout of evidence, when it is generated text like the rest of the answer."}]},"prev":{"slug":"why-answers-differ","title":"Why the same question gives different answers","url":"https://addaly.com/learn/ai-actually-explained/why-answers-differ"},"next":{"slug":"taught-to-be-helpful","title":"The second training, where the assistant is made","url":"https://addaly.com/learn/ai-actually-explained/taught-to-be-helpful"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What was in the pile', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/what-it-read (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What was in the pile","courseTitle":"AI, Actually Explained","module":{"index":3,"slug":"how-a-chatbot-works","title":"What a chatbot is actually doing"},"answer":null,"keyIdea":"A model is a compression of what was written down, in public, in that language, before the cut-off, and its fluency stays uniform even where its knowledge is thin.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-pile-decides-the-model","runs":[{"text":"The pile decides the model"}]},{"t":"p","runs":[{"text":"A model is a compression of what it was trained on. So the honest way to predict what it will be good at is to ask what was in the pile."}]},{"t":"p","runs":[{"text":"For a large language model the pile is text on the order of ten to fifteen trillion tokens: a quantity nobody could read, roughly equivalent to tens of millions of books. It is assembled from a handful of recognisable sources:"}]},{"t":"ul","items":[[{"text":"Web crawl.","b":true},{"text":" The bulk of it. Common Crawl, a free public archive of scraped web pages, is the starting point for most open efforts, filtered heavily."}],[{"text":"Wikipedia","b":true},{"text":", in many languages, weighted far above its size because it is clean and factual."}],[{"text":"Books","b":true},{"text":", from public-domain collections and, contentiously, from other sources."}],[{"text":"Code","b":true},{"text":", largely from public repositories on GitHub. Training on code appears to improve reasoning on non-code tasks, which was not the plan."}],[{"text":"Forums and question sites","b":true},{"text":" (Reddit, Stack Exchange) which teach conversational structure and the shape of an answer."}],[{"text":"Scientific papers","b":true},{"text":", from arXiv and PubMed."}],[{"text":"Increasingly, "},{"text":"synthetic text","b":true},{"text":" generated by other models."}]]},{"t":"h2","id":"what-is-not-in-the-pile","runs":[{"text":"What is not in the pile"}]},{"t":"p","runs":[{"text":"More useful than the list is its shadow."}]},{"t":"p","runs":[{"text":"Anything not written down.","b":true},{"text":" Craft knowledge that passes by demonstration (how a fabric should feel, how an engine sounds when the bearing is going) is thinly represented, because people who know it write it down rarely."}]},{"t":"p","runs":[{"text":"Anything behind a wall.","b":true},{"text":" Textbooks in copyright, most journalism after a paywall, court records that are not digitised, internal company documents, most of WhatsApp and every private conversation."}]},{"t":"p","runs":[{"text":"Anything in a language with a small web presence.","b":true},{"text":" This is the big one for most of the world. English is perhaps five per cent of humanity and a large majority of the training text. A language with a hundred million speakers and a thin internet is a language the model is weak in, and it will not say so."}]},{"t":"p","runs":[{"text":"Anything from before the internet that nobody scanned.","b":true},{"text":" Local history, regional legal practice, oral tradition."}]},{"t":"p","runs":[{"text":"Anything after the cut-off","b":true},{"text":", which has its own lesson."}]},{"t":"h2","id":"filtering-and-what-filtering-does","runs":[{"text":"Filtering, and what filtering does"}]},{"t":"p","runs":[{"text":"Raw crawl is largely unusable: boilerplate, navigation, spam, duplicates, machine-translated sludge. So it is filtered: deduplicated, scored by quality classifiers, and screened for certain categories of harmful content."}]},{"t":"p","runs":[{"text":"Deduplication matters more than it sounds. Training on near-identical copies of the same page makes memorisation likelier and generalisation worse, so labs spend serious effort on it."}]},{"t":"p","runs":[{"text":"But a quality classifier is itself a model with a view about what good text looks like, and that view was formed from examples somebody chose. Filtering is not neutral, and there is documented evidence that some filters disproportionately remove text written in non-standard dialects. Cleaning a corpus is an editorial act."}]},{"t":"h2","id":"what-is-knowable-and-what-is-now-secret","runs":[{"text":"What is knowable, and what is now secret"}]},{"t":"p","runs":[{"text":"This is worth being blunt about. Until around 2022, papers described training data in reasonable detail. Now the leading commercial labs do not disclose theirs, for competitive and legal reasons. When someone tells you what GPT-class models were trained on, they are inferring."}]},{"t":"p","runs":[{"text":"Open alternatives exist and are valuable if you want to see inside one:"}]},{"t":"ul","items":[[{"text":"OLMo","b":true},{"text":", from the Allen Institute for AI, ships with its full training corpus, "},{"text":"Dolma","code":true},{"text":", published and downloadable."}],[{"text":"The Pile","b":true},{"text":" and "},{"text":"RedPajama","b":true},{"text":" are open, documented datasets you can inspect."}],[{"text":"Pythia","b":true},{"text":" is a set of models released with their data and intermediate checkpoints specifically so researchers can study what training does."}]]},{"t":"p","runs":[{"text":"If you ever want to move from believing things about training data to checking them, those are the doors."}]},{"t":"h2","id":"the-one-habit-to-take-away","runs":[{"text":"The one habit to take away"}]},{"t":"p","runs":[{"text":"Before trusting a model on a subject, ask: "},{"text":"was this written down, in quantity, in public, in this language, before the cut-off?","b":true}]},{"t":"p","runs":[{"text":"Four yeses and the model is probably solid. A no on any of them and you should expect confident output of unknown quality, because the model’s fluency is uniform, and its knowledge is not."}]},{"t":"p","runs":[{"text":"That asymmetry, between even fluency and uneven knowledge, is the single most dangerous property of these systems, and it comes directly from the pile."}]}],"check":{"question":[{"text":"A user asks a model detailed questions about tenancy customs in a small district, and gets fluent, specific, confident answers. What is the most reasonable stance?"}],"options":[{"text":[{"text":"The fluency indicates the model has relevant training material, since it would hedge if it did not."}],"correct":false,"tempting":[{"text":"Assumes fluency tracks knowledge; the model’s fluency is uniform and does not degrade where its material is thin."}]},{"text":[{"text":"Local legal detail is rarely written up in quantity online, so specificity here is a warning rather than a reassurance, and each claim needs a source."}],"correct":true,"tempting":null},{"text":[{"text":"The answers are likely drawn from national law, which is well documented, so they will be broadly right."}],"correct":false,"tempting":[{"text":"Guesses at a plausible substitution the model might make, when what the model produces is a generated blend rather than a documented fallback."}]},{"text":[{"text":"Asking the model to state its confidence would separate the well-supported claims from the invented ones."}],"correct":false,"tempting":[{"text":"Treats stated confidence as a readout of evidence, when it is generated text like the rest of the answer."}]}],"order":[3,1,0,2],"why":null},"prev":{"slug":"why-answers-differ","title":"Why the same question gives different answers","minutes":8},"next":{"slug":"taught-to-be-helpful","title":"The second training, where the assistant is made","minutes":9}}}