{"course":{"slug":"evaluating-ai","title":"Knowing If It Works","url":"https://addaly.com/learn/evaluating-ai","level":"intermediate","track":"build"},"lesson":{"slug":"splits-and-leakage","title":"Two sets, and the ways they get contaminated","url":"https://addaly.com/learn/evaluating-ai/splits-and-leakage","position":8,"of":76,"minutes":9,"answer":"Keep a development set you tune against and a test set you almost never look at, split by group and by time, because a set you have optimised against no longer predicts anything.","keyIdea":"Keep a development set you tune against and a test set you almost never look at, split by group and by time, because a set you have optimised against no longer predicts anything."},"body":{"format":"markdown","text":"## Why one set is not enough\n\nYou will look at your evaluation set constantly. You will read the failures, change the prompt to fix them, re-run, read the new failures, change it again. After twenty rounds of that, the score on that set is no longer an estimate of how the system behaves on new inputs. It is a record of how well you fitted a hundred specific examples, using your own brain as the optimiser.\n\nThis is overfitting, and it does not require a training loop. It happens to prompts, to retrieval configurations, to threshold choices, to anything you tune while watching a number.\n\nThe fix is two sets, with different rules:\n\n- **The development set.** Look at it as often as you like. Read every failure. Tune against it. This is where the work happens.\n- **The test set.** Do not read the items. Run it when a decision is being made (before a release, before telling somebody a number) and then leave it alone. Once a month is generous. Once a quarter is common.\n\nSplit them before you start, by a rule that cannot drift: a hash of the item id, `int(md5(id)[:8], 16) % 10 < 2` for a 20% test set. Written that way, adding new items later keeps the split stable and nobody has to remember anything.\n\nIf the test number is much worse than the dev number, you have not been improving the system. You have been improving your fit to the dev set. That gap is the most useful diagnostic in this course and it is invisible with one set.\n\n## Near-duplicates are the commonest leak\n\nSuppose your data comes from support tickets. \"How do I reset my password\" appears 340 times in slightly different wording. Sample randomly into two sets and near-identical items land on both sides. Your test set is now measuring the same question your dev set already taught you to answer, and it will look wonderful.\n\nOne team building an FAQ router reported 94% on a held-out set and 61% in production. Thirty of their hundred held-out items were paraphrases of items in the training prompt's few-shot examples. The set was not testing generalisation at all.\n\nDedupe before splitting. A cheap and surprisingly effective pass: normalise (lowercase, strip punctuation, collapse whitespace), then drop exact hash matches; then embed everything with a small local model such as `all-MiniLM-L6-v2` through `sentence-transformers` (free, runs on a laptop CPU) and drop anything with cosine similarity above about 0.95 to an item already kept. Look at twenty of the pairs you dropped before trusting the threshold.\n\n## Split by group, not by row\n\nRows are usually not independent. Twelve tickets from the same customer share vocabulary, product, and often the same underlying bug. Ten pages from the same document share formatting. Put some in dev and some in test and information crosses the wall.\n\nSo split by the unit that shares information: customer, document, seller, conversation, patient. All rows for one group go entirely to one side. `GroupKFold` in scikit-learn does this in one line, and doing it by hand in SQL is a `GROUP BY` and a hash.\n\n## Split by time when time matters\n\nIf what you are building will run on next month's data, evaluating on a random sample of last year's is optimistic. Language, product names, scam patterns and customer questions all move. Take the last four weeks as the test set and everything earlier as dev. The number will be lower than a random split gives you. The lower number is the honest one.\n\nThis also catches an error that random splits hide entirely: features computed with information that would not exist at prediction time. If your input includes a field the system only learns *after* the event you are predicting, a random split rewards it and a time split exposes it.\n\n## Benchmark contamination\n\nThere is a version of this problem you cannot fix by splitting, because the leak happened before you arrived. Public evaluation sets (the ones a model's card quotes) are on the open web, and models are trained on the open web. A model may have seen the questions and the answers.\n\nThe signs are recognisable: near-perfect scores on an old public benchmark alongside ordinary performance on a freshly written variant of the same task; sensitivity to trivial rewording that should not matter; occasionally, verbatim recall of a question's phrasing. Some benchmark authors embed *canary strings*, unique identifiers that should never appear in training data, so contamination can be detected after the fact. It usually is not checked.\n\nThe practical consequence is narrow and important: your own set, made from your own data, after your model's training cutoff, is the only set you can be confident is uncontaminated. That is a large part of why lesson two asks you to build one by hand.\n\n## The rule that keeps it working\n\nWrite the rules down where the set lives: which split is which, who may look at what, and how often the test set is run. Then treat a test-set peek the way you would treat a schema migration: not forbidden, but noticed, logged and deliberate.\n\nThe first time somebody quietly tunes against the test set to make a release, the set stops being evidence, and nothing in the output tells you that it has happened."},"check":{"question":"A team building a document-classification feature splits 4,000 pages randomly into dev and test. Both sets score around 93%, but live accuracy is near 70%. The pages came from 180 source documents. What is the most likely explanation?","options":[{"text":"The 4,000 pages are too few to estimate accuracy, so both numbers are noise.","correct":false,"tempting":"Blames sample size when 4,000 items give an interval of roughly a point, far too tight to explain a 23-point gap."},{"text":"The test set was run too often, so the team has overfitted to it as well as to dev.","correct":false,"tempting":"Names a real hazard, but repeated test runs would push dev and test apart, not push both above live performance together."},{"text":"Live traffic is simply harder than the collected data, and nothing about the split is at fault.","correct":false,"tempting":"Accepts a distribution-shift story without checking the structural flaw sitting in plain sight in the way the split was made."},{"text":"Pages from the same document landed on both sides of the split, so the test set was measuring documents the system had effectively already been tuned on.","correct":true}]},"prev":{"slug":"synthetic-test-cases","title":"Test cases you generate, and what they cannot tell you","url":"https://addaly.com/learn/evaluating-ai/synthetic-test-cases"},"next":{"slug":"baselines-and-ceilings","title":"The number above it and the number below it","url":"https://addaly.com/learn/evaluating-ai/baselines-and-ceilings"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Two sets, and the ways they get contaminated', in Knowing If It Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/evaluating-ai/splits-and-leakage (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Two sets, and the ways they get contaminated","courseTitle":"Knowing If It Works","module":{"index":1,"slug":"evidence","title":"What counts as evidence"},"answer":null,"keyIdea":"Keep a development set you tune against and a test set you almost never look at, split by group and by time, because a set you have optimised against no longer predicts anything.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-one-set-is-not-enough","runs":[{"text":"Why one set is not enough"}]},{"t":"p","runs":[{"text":"You will look at your evaluation set constantly. You will read the failures, change the prompt to fix them, re-run, read the new failures, change it again. After twenty rounds of that, the score on that set is no longer an estimate of how the system behaves on new inputs. It is a record of how well you fitted a hundred specific examples, using your own brain as the optimiser."}]},{"t":"p","runs":[{"text":"This is overfitting, and it does not require a training loop. It happens to prompts, to retrieval configurations, to threshold choices, to anything you tune while watching a number."}]},{"t":"p","runs":[{"text":"The fix is two sets, with different rules:"}]},{"t":"ul","items":[[{"text":"The development set.","b":true},{"text":" Look at it as often as you like. Read every failure. Tune against it. This is where the work happens."}],[{"text":"The test set.","b":true},{"text":" Do not read the items. Run it when a decision is being made (before a release, before telling somebody a number) and then leave it alone. Once a month is generous. Once a quarter is common."}]]},{"t":"p","runs":[{"text":"Split them before you start, by a rule that cannot drift: a hash of the item id, "},{"text":"int(md5(id)[:8], 16) % 10 < 2","code":true},{"text":" for a 20% test set. Written that way, adding new items later keeps the split stable and nobody has to remember anything."}]},{"t":"p","runs":[{"text":"If the test number is much worse than the dev number, you have not been improving the system. You have been improving your fit to the dev set. That gap is the most useful diagnostic in this course and it is invisible with one set."}]},{"t":"h2","id":"near-duplicates-are-the-commonest-leak","runs":[{"text":"Near-duplicates are the commonest leak"}]},{"t":"p","runs":[{"text":"Suppose your data comes from support tickets. “How do I reset my password” appears 340 times in slightly different wording. Sample randomly into two sets and near-identical items land on both sides. Your test set is now measuring the same question your dev set already taught you to answer, and it will look wonderful."}]},{"t":"p","runs":[{"text":"One team building an FAQ router reported 94% on a held-out set and 61% in production. Thirty of their hundred held-out items were paraphrases of items in the training prompt’s few-shot examples. The set was not testing generalisation at all."}]},{"t":"p","runs":[{"text":"Dedupe before splitting. A cheap and surprisingly effective pass: normalise (lowercase, strip punctuation, collapse whitespace), then drop exact hash matches; then embed everything with a small local model such as "},{"text":"all-MiniLM-L6-v2","code":true},{"text":" through "},{"text":"sentence-transformers","code":true},{"text":" (free, runs on a laptop CPU) and drop anything with cosine similarity above about 0.95 to an item already kept. Look at twenty of the pairs you dropped before trusting the threshold."}]},{"t":"h2","id":"split-by-group-not-by-row","runs":[{"text":"Split by group, not by row"}]},{"t":"p","runs":[{"text":"Rows are usually not independent. Twelve tickets from the same customer share vocabulary, product, and often the same underlying bug. Ten pages from the same document share formatting. Put some in dev and some in test and information crosses the wall."}]},{"t":"p","runs":[{"text":"So split by the unit that shares information: customer, document, seller, conversation, patient. All rows for one group go entirely to one side. "},{"text":"GroupKFold","code":true},{"text":" in scikit-learn does this in one line, and doing it by hand in SQL is a "},{"text":"GROUP BY","code":true},{"text":" and a hash."}]},{"t":"h2","id":"split-by-time-when-time-matters","runs":[{"text":"Split by time when time matters"}]},{"t":"p","runs":[{"text":"If what you are building will run on next month’s data, evaluating on a random sample of last year’s is optimistic. Language, product names, scam patterns and customer questions all move. Take the last four weeks as the test set and everything earlier as dev. The number will be lower than a random split gives you. The lower number is the honest one."}]},{"t":"p","runs":[{"text":"This also catches an error that random splits hide entirely: features computed with information that would not exist at prediction time. If your input includes a field the system only learns "},{"text":"after","i":true},{"text":" the event you are predicting, a random split rewards it and a time split exposes it."}]},{"t":"h2","id":"benchmark-contamination","runs":[{"text":"Benchmark contamination"}]},{"t":"p","runs":[{"text":"There is a version of this problem you cannot fix by splitting, because the leak happened before you arrived. Public evaluation sets (the ones a model’s card quotes) are on the open web, and models are trained on the open web. A model may have seen the questions and the answers."}]},{"t":"p","runs":[{"text":"The signs are recognisable: near-perfect scores on an old public benchmark alongside ordinary performance on a freshly written variant of the same task; sensitivity to trivial rewording that should not matter; occasionally, verbatim recall of a question’s phrasing. Some benchmark authors embed "},{"text":"canary strings","i":true},{"text":", unique identifiers that should never appear in training data, so contamination can be detected after the fact. It usually is not checked."}]},{"t":"p","runs":[{"text":"The practical consequence is narrow and important: your own set, made from your own data, after your model’s training cutoff, is the only set you can be confident is uncontaminated. That is a large part of why lesson two asks you to build one by hand."}]},{"t":"h2","id":"the-rule-that-keeps-it-working","runs":[{"text":"The rule that keeps it working"}]},{"t":"p","runs":[{"text":"Write the rules down where the set lives: which split is which, who may look at what, and how often the test set is run. Then treat a test-set peek the way you would treat a schema migration: not forbidden, but noticed, logged and deliberate."}]},{"t":"p","runs":[{"text":"The first time somebody quietly tunes against the test set to make a release, the set stops being evidence, and nothing in the output tells you that it has happened."}]}],"check":{"question":[{"text":"A team building a document-classification feature splits 4,000 pages randomly into dev and test. Both sets score around 93%, but live accuracy is near 70%. The pages came from 180 source documents. What is the most likely explanation?"}],"options":[{"text":[{"text":"The 4,000 pages are too few to estimate accuracy, so both numbers are noise."}],"correct":false,"tempting":[{"text":"Blames sample size when 4,000 items give an interval of roughly a point, far too tight to explain a 23-point gap."}]},{"text":[{"text":"The test set was run too often, so the team has overfitted to it as well as to dev."}],"correct":false,"tempting":[{"text":"Names a real hazard, but repeated test runs would push dev and test apart, not push both above live performance together."}]},{"text":[{"text":"Live traffic is simply harder than the collected data, and nothing about the split is at fault."}],"correct":false,"tempting":[{"text":"Accepts a distribution-shift story without checking the structural flaw sitting in plain sight in the way the split was made."}]},{"text":[{"text":"Pages from the same document landed on both sides of the split, so the test set was measuring documents the system had effectively already been tuned on."}],"correct":true,"tempting":null}],"order":[2,0,1,3],"why":null},"prev":{"slug":"synthetic-test-cases","title":"Test cases you generate, and what they cannot tell you","minutes":8},"next":{"slug":"baselines-and-ceilings","title":"The number above it and the number below it","minutes":9}}}