{"course":{"slug":"ai-and-you","title":"AI, Safety and What Goes Wrong","url":"https://addaly.com/learn/ai-and-you","level":"beginner","track":"start"},"lesson":{"slug":"training-data-is-a-choice","title":"Where the data came from","url":"https://addaly.com/learn/ai-and-you/training-data-is-a-choice","position":3,"of":73,"minutes":3,"answer":"Training data is a stack of human choices, and each one shows in the answers.","keyIdea":"Training data is a stack of human choices, and each one shows in the answers."},"body":{"format":"markdown","text":"## Forty-five terabytes in, 570 gigabytes kept\n\nIn 2020 OpenAI published the recipe for GPT-3, the model family ChatGPT grew from. Its largest ingredient was Common Crawl, a free archive of the web. The team took 41 monthly snapshots from 2016 to 2019: 45 terabytes of compressed text. After filtering, they kept 570 gigabytes. Roughly one part in 80 survived.\n\n\"Trained on the internet\" sounds like a fact of nature, like rain. It describes a chain of decisions, and the first one was which part in 80 to keep.\n\n## How \"good\" was defined\n\nThe team trained a classifier to tell high-quality pages from raw crawl. For its examples of quality they used WebText, a collection of pages linked from Reddit posts, plus Wikipedia and a books collection. Pages that resembled those were kept.\n\nSo \"good writing\" meant, in practice, \"looks like what one English-speaking online community linked to, plus an encyclopedia\". Every answer the model gives inherits that definition.\n\n## How often each source was seen\n\nNext the team chose how often to show each source. Their paper says the mix was deliberately not in proportion to size.\n\n> Figure: GPT-3, 2020: how many times each source was seen. Bars (times seen in training): Wikipedia, 3 billion tokens 3.4; WebText2, 19 billion tokens 2.9; Books1, 12 billion tokens 1.9; Common Crawl, filtered, 410 billion tokens 0.44; Books2, 55 billion tokens 0.43. From Table 2.2 of the GPT-3 paper. The team sampled by judged quality, not by size, so an encyclopedia shaped the model far more per word than the filtered web did.\n\nWikipedia, 3 billion tokens, was seen 3.4 times. The filtered web, 410 billion tokens, was seen 0.44 times: more than half of it never came up at all.\n\n## What the choices do to you\n\n**Language.** By word count, 93% of GPT-3's data was English. A model that answers well in English and worse in Marathi or Bengali saw far less of those languages.\n\n**Time.** Every model has a cutoff. Ask about something after it and you get a refusal, a search result, or a confident answer built from the shape of older events. The third is the dangerous one.\n\n**Who writes online.** Web text over-represents people with the time, connection and confidence to publish. The model's sense of what is normal comes from them.\n\n## Where you can look\n\nSome training sets are public. C4, The Pile, Dolma and FineWeb publish what they contain, and some let you search for a document. For most commercial models you cannot see the mix. The EU AI Act now requires providers of general-purpose models to publish a sufficiently detailed summary of the content used for training.\n\nWhen someone says a model was \"trained on all of human knowledge\", the useful reply is to ask which crawl it came from, how it was filtered, in what proportions it was mixed, and up to what date."},"check":{"question":"An assistant gives noticeably weaker answers in Bengali than in English, on a subject where good Bengali material exists. What is the likeliest explanation?","options":[{"text":"Its architecture cannot process non-Latin scripts accurately","correct":false,"tempting":"Non-Latin scripts cost more tokens, which is real. Cost is not accuracy, and the accuracy gap tracks how much data there was."},{"text":"Safety filters tuned for English strip detail from Bengali","correct":false,"tempting":"Filters produce refusals and hedges. They do not make subject knowledge subtly thinner."},{"text":"It translates to English inside, then back, and loses detail","correct":false,"tempting":"An intuitive pipeline, but a multilingual model has no separate translation step to lose information in."},{"text":"Bengali was a thin slice of the data it was trained on","correct":true}],"why":"Somebody chose the mix. GPT-3's data was 93% English by word count, and thinner data gives thinner answers."},"prev":{"slug":"what-the-machine-is-doing","title":"What the model computes","url":"https://addaly.com/learn/ai-and-you/what-the-machine-is-doing"},"next":{"slug":"the-benchmark-and-the-world","title":"What a benchmark score leaves out","url":"https://addaly.com/learn/ai-and-you/the-benchmark-and-the-world"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Source and assessment review continues. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Where the data came from', in AI, Safety and What Goes Wrong. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-and-you/training-data-is-a-choice (Accessed: 21 November 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Where the data came from","courseTitle":"AI, Safety and What Goes Wrong","module":{"index":1,"slug":"how-things-fail","title":"How these systems fail"},"answer":null,"keyIdea":"Training data is a stack of human choices, and each one shows in the answers.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"caption":"From Table 2.2 of the GPT-3 paper. The team sampled by judged quality, not by size, so an encyclopedia shaped the model far more per word than the filtered web did.","series":[{"highlight":true,"label":"Wikipedia, 3 billion tokens","value":3.4},{"label":"WebText2, 19 billion tokens","value":2.9},{"label":"Books1, 12 billion tokens","value":1.9},{"label":"Common Crawl, filtered, 410 billion tokens","value":0.44},{"label":"Books2, 55 billion tokens","value":0.43}],"title":"GPT-3, 2020: how many times each source was seen","type":"bars","unit":"times seen in training"},"alt":"Figure: GPT-3, 2020: how many times each source was seen. Bars (times seen in training): Wikipedia, 3 billion tokens 3.4; WebText2, 19 billion tokens 2.9; Books1, 12 billion tokens 1.9; Common Crawl, filtered, 410 billion tokens 0.44; Books2, 55 billion tokens 0.43. From Table 2.2 of the GPT-3 paper. The team sampled by judged quality, not by size, so an encyclopedia shaped the model far more per word than the filtered web did."}},"blocks":[{"t":"h2","id":"forty-five-terabytes-in-570-gigabytes-kept","runs":[{"text":"Forty-five terabytes in, 570 gigabytes kept"}]},{"t":"p","runs":[{"text":"In 2020 OpenAI published the recipe for GPT-3, the model family ChatGPT grew from. Its largest ingredient was Common Crawl, a free archive of the web. The team took 41 monthly snapshots from 2016 to 2019: 45 terabytes of compressed text. After filtering, they kept 570 gigabytes. Roughly one part in 80 survived."}]},{"t":"quick","prompt":"Your call: who decided which part of the web to keep?","options":["The web’s own search rankings","A classifier the team trained","A random sample"],"answer":1,"reveal":"A classifier the team trained. The next section is what it was trained to prefer."},{"t":"p","runs":[{"text":"“Trained on the internet” sounds like a fact of nature, like rain. It describes a chain of decisions, and the first one was which part in 80 to keep."}]},{"t":"h2","id":"how-good-was-defined","runs":[{"text":"How “good” was defined"}]},{"t":"p","runs":[{"text":"The team trained a classifier to tell high-quality pages from raw crawl. For its examples of quality they used WebText, a collection of pages linked from Reddit posts, plus Wikipedia and a books collection. Pages that resembled those were kept."}]},{"t":"p","runs":[{"text":"So “good writing” meant, in practice, “looks like what one English-speaking online community linked to, plus an encyclopedia”. Every answer the model gives inherits that definition."}]},{"t":"h2","id":"how-often-each-source-was-seen","runs":[{"text":"How often each source was seen"}]},{"t":"p","runs":[{"text":"Next the team chose how often to show each source. Their paper says the mix was deliberately not in proportion to size."}]},{"t":"p","runs":[{"text":"Wikipedia, 3 billion tokens, was seen 3.4 times. The filtered web, 410 billion tokens, was seen 0.44 times: more than half of it never came up at all."}]},{"t":"h2","id":"what-the-choices-do-to-you","runs":[{"text":"What the choices do to you"}]},{"t":"p","runs":[{"text":"Language.","b":true},{"text":" By word count, 93% of GPT-3’s data was English. A model that answers well in English and worse in Marathi or Bengali saw far less of those languages."}]},{"t":"p","runs":[{"text":"Time.","b":true},{"text":" Every model has a cutoff. Ask about something after it and you get a refusal, a search result, or a confident answer built from the shape of older events. The third is the dangerous one."}]},{"t":"p","runs":[{"text":"Who writes online.","b":true},{"text":" Web text over-represents people with the time, connection and confidence to publish. The model’s sense of what is normal comes from them."}]},{"t":"h2","id":"where-you-can-look","runs":[{"text":"Where you can look"}]},{"t":"p","runs":[{"text":"Some training sets are public. C4, The Pile, Dolma and FineWeb publish what they contain, and some let you search for a document. For most commercial models you cannot see the mix. The EU AI Act now requires providers of general-purpose models to publish a sufficiently detailed summary of the content used for training."}]},{"t":"p","runs":[{"text":"When someone says a model was “trained on all of human knowledge”, the useful reply is to ask which crawl it came from, how it was filtered, in what proportions it was mixed, and up to what date."}]}],"check":{"question":[{"text":"An assistant gives noticeably weaker answers in Bengali than in English, on a subject where good Bengali material exists. What is the likeliest explanation?"}],"options":[{"text":[{"text":"Its architecture cannot process non-Latin scripts accurately"}],"correct":false,"tempting":[{"text":"Non-Latin scripts cost more tokens, which is real. Cost is not accuracy, and the accuracy gap tracks how much data there was."}]},{"text":[{"text":"Safety filters tuned for English strip detail from Bengali"}],"correct":false,"tempting":[{"text":"Filters produce refusals and hedges. They do not make subject knowledge subtly thinner."}]},{"text":[{"text":"It translates to English inside, then back, and loses detail"}],"correct":false,"tempting":[{"text":"An intuitive pipeline, but a multilingual model has no separate translation step to lose information in."}]},{"text":[{"text":"Bengali was a thin slice of the data it was trained on"}],"correct":true,"tempting":null}],"order":[1,2,0,3],"why":[{"text":"Somebody chose the mix. GPT-3’s data was 93% English by word count, and thinner data gives thinner answers."}]},"prev":{"slug":"what-the-machine-is-doing","title":"What the model computes","minutes":4},"next":{"slug":"the-benchmark-and-the-world","title":"What a benchmark score leaves out","minutes":3}}}