{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"contamination-in-your-own-data","title":"Contamination you can check, and contamination you cannot","url":"https://addaly.com/learn/fine-tuning/contamination-in-your-own-data","position":72,"of":84,"minutes":9,"answer":"Thirteen-gram overlap will find contamination between your training and evaluation sets, and the answer-ordering test can detect memorised benchmarks, but the base model's pretraining corpus is unchecked and unknowable.","keyIdea":"Thirteen-gram overlap will find contamination between your training and evaluation sets, and the answer-ordering test can detect memorised benchmarks, but the base model's pretraining corpus is unchecked and unknowable, which is why a held-out set built after the training cutoff is the only evidence that holds."},"body":{"format":"markdown","text":"## Two kinds, and only one is yours to fix\n\n**The kind you can check:** your training data contains items from the evaluation you are quoting. This happens more easily than people expect, particularly when your training data is synthetic: a teacher model that memorised a public benchmark during pretraining will happily reproduce its items when asked to generate examples of that kind of task.\n\n**The kind you cannot check:** the base model's pretraining corpus contained the benchmark. You have no access to that corpus, no way to search it, and no way to rule it out.\n\nThe second is why public benchmark numbers on a fine-tuned model are weak evidence about anything, and why your own held-out set (built from your own data, after your own cutoff) is the strong evidence.\n\n## Checking the kind you can\n\nThe conventional test is n-gram overlap, and the convention from the large pretraining papers is 13-grams: if a 13-word sequence from an evaluation item appears anywhere in the training data, treat the item as contaminated.\n\n```python\ndef ngrams(text, n=13):\n    w = text.lower().split()\n    return {\" \".join(w[i:i+n]) for i in range(max(0, len(w)-n+1))}\n\ntrain_grams = set().union(*(ngrams(t) for t in train_texts))\nhits = [i for i, t in enumerate(eval_texts) if ngrams(t) & train_grams]\nprint(f\"{len(hits)}/{len(eval_texts)} eval items share a 13-gram with training\")\n```\n\nThirteen is a reasonable default: long enough that ordinary phrases do not match, short enough to catch a paraphrase that kept one clause. Run it in both directions: evaluation against training, and training against any public benchmark you intend to quote.\n\nSome benchmarks embed a canary string specifically so that their presence in a corpus can be detected. Grep for it. If it is in your training data, something upstream scraped the benchmark.\n\n## The ordering test\n\nA cleverer check for multiple-choice benchmarks, and it needs no access to anyone's corpus.\n\nIf a model has memorised a question as it appears in the benchmark, it will assign higher probability to the *canonical ordering* of its answer options than to a shuffled ordering of the same options. A model that is reasoning about the question has no reason to prefer one arrangement.\n\n```python\np_original = seq_logprob(model, render(q, q.options))\np_shuffled = mean(seq_logprob(model, render(q, shuffle(q.options)))\n                  for _ in range(5))\n```\n\nA consistent preference for the original ordering across many questions is evidence the model saw the benchmark in that form. This is a real and usable diagnostic, and it is one of the few tools available for the contamination you cannot otherwise inspect.\n\n## What contamination does to your decisions\n\nIt inflates the number you are using to make a decision, and it inflates it unevenly. The items that leaked are the ones the model scores well on, so the headline rises while the model's actual generalisation has not moved. You then ship, and production disagrees.\n\nThe specific damage in a fine-tuning project: you compare a tuned model against a baseline, and the tuned model's *training data* overlaps the evaluation while the baseline's does not. The comparison is structurally unfair in the tuned model's favour, and the gap you measure is partly recall.\n\n## The defences\n\n1. **Build the evaluation set from data the training set cannot contain**: a later time period, a held-out group, or items written specifically for evaluation and never released into the training pipeline.\n2. **Run the n-gram check** across your split, and report the number in your evaluation write-up even when it is zero. A stated zero is information; an unstated one is an assumption.\n3. **Do not quote public benchmarks** as evidence that your fine-tune worked. They cannot be cleaned of base-model contamination, and they are not measuring your task anyway.\n4. **Keep a locked set** created after the training data's cutoff date. Temporal separation is the one defence that also protects against the contamination you cannot inspect, because a benchmark that did not exist when the base model was trained cannot be in its corpus.\n\n## The summary\n\nYou can rule out contamination between your training set and your evaluation set. You cannot rule it out between the base model's pretraining and any public benchmark. Say which of the two you have checked, and do not let a reader assume you checked both. That sentence, written into an evaluation report, is worth more than an additional decimal place on any of its numbers."},"check":{"question":"A tuned model scores markedly higher on a public multiple-choice benchmark when the answer options are in their canonical order than when the same options are shuffled. What does this indicate?","options":[{"text":"The model has seen those questions in that exact arrangement, so its score reflects recall.","correct":true},{"text":"The fine-tuning data taught a preference for the benchmark's formatting conventions, which shuffling disrupts.","correct":false,"tempting":"Formatting is unchanged by shuffling the options; only their arrangement differs, so a formatting preference cannot explain the gap."},{"text":"Position bias in the model, which prefers the first listed option regardless of content.","correct":false,"tempting":"Names a real bias, but position bias would affect canonical and shuffled orderings alike rather than distinguishing them."},{"text":"The shuffled version is genuinely harder, because reordering breaks the logical progression the question author intended.","correct":false,"tempting":"Assumes option order carries meaning the model exploits legitimately, which would not produce a consistent gap across unrelated questions."}]},"prev":{"slug":"judging-a-model-you-distilled","title":"Judging a model you distilled from the judge","url":"https://addaly.com/learn/fine-tuning/judging-a-model-you-distilled"},"next":{"slug":"safety-regression-after-tuning","title":"Safety regression, including from benign data","url":"https://addaly.com/learn/fine-tuning/safety-regression-after-tuning"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Contamination you can check, and contamination you cannot', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/contamination-in-your-own-data (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Contamination you can check, and contamination you cannot","courseTitle":"Fine-Tuning, and When Not To","module":{"index":8,"slug":"did-it-actually-work","title":"Did it actually work"},"answer":null,"keyIdea":"Thirteen-gram overlap will find contamination between your training and evaluation sets, and the answer-ordering test can detect memorised benchmarks, but the base model’s pretraining corpus is unchecked and unknowable, which is why a held-out set built after the training cutoff is the only evidence that holds.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-kinds-and-only-one-is-yours-to-fix","runs":[{"text":"Two kinds, and only one is yours to fix"}]},{"t":"p","runs":[{"text":"The kind you can check:","b":true},{"text":" your training data contains items from the evaluation you are quoting. This happens more easily than people expect, particularly when your training data is synthetic: a teacher model that memorised a public benchmark during pretraining will happily reproduce its items when asked to generate examples of that kind of task."}]},{"t":"p","runs":[{"text":"The kind you cannot check:","b":true},{"text":" the base model’s pretraining corpus contained the benchmark. You have no access to that corpus, no way to search it, and no way to rule it out."}]},{"t":"p","runs":[{"text":"The second is why public benchmark numbers on a fine-tuned model are weak evidence about anything, and why your own held-out set (built from your own data, after your own cutoff) is the strong evidence."}]},{"t":"h2","id":"checking-the-kind-you-can","runs":[{"text":"Checking the kind you can"}]},{"t":"p","runs":[{"text":"The conventional test is n-gram overlap, and the convention from the large pretraining papers is 13-grams: if a 13-word sequence from an evaluation item appears anywhere in the training data, treat the item as contaminated."}]},{"t":"code","lang":"python","label":"Python","text":"def ngrams(text, n=13):\n    w = text.lower().split()\n    return {\" \".join(w[i:i+n]) for i in range(max(0, len(w)-n+1))}\n\ntrain_grams = set().union(*(ngrams(t) for t in train_texts))\nhits = [i for i, t in enumerate(eval_texts) if ngrams(t) & train_grams]\nprint(f\"{len(hits)}/{len(eval_texts)} eval items share a 13-gram with training\")"},{"t":"p","runs":[{"text":"Thirteen is a reasonable default: long enough that ordinary phrases do not match, short enough to catch a paraphrase that kept one clause. Run it in both directions: evaluation against training, and training against any public benchmark you intend to quote."}]},{"t":"p","runs":[{"text":"Some benchmarks embed a canary string specifically so that their presence in a corpus can be detected. Grep for it. If it is in your training data, something upstream scraped the benchmark."}]},{"t":"h2","id":"the-ordering-test","runs":[{"text":"The ordering test"}]},{"t":"p","runs":[{"text":"A cleverer check for multiple-choice benchmarks, and it needs no access to anyone’s corpus."}]},{"t":"p","runs":[{"text":"If a model has memorised a question as it appears in the benchmark, it will assign higher probability to the "},{"text":"canonical ordering","i":true},{"text":" of its answer options than to a shuffled ordering of the same options. A model that is reasoning about the question has no reason to prefer one arrangement."}]},{"t":"code","lang":"python","label":"Python","text":"p_original = seq_logprob(model, render(q, q.options))\np_shuffled = mean(seq_logprob(model, render(q, shuffle(q.options)))\n                  for _ in range(5))"},{"t":"p","runs":[{"text":"A consistent preference for the original ordering across many questions is evidence the model saw the benchmark in that form. This is a real and usable diagnostic, and it is one of the few tools available for the contamination you cannot otherwise inspect."}]},{"t":"h2","id":"what-contamination-does-to-your-decisions","runs":[{"text":"What contamination does to your decisions"}]},{"t":"p","runs":[{"text":"It inflates the number you are using to make a decision, and it inflates it unevenly. The items that leaked are the ones the model scores well on, so the headline rises while the model’s actual generalisation has not moved. You then ship, and production disagrees."}]},{"t":"p","runs":[{"text":"The specific damage in a fine-tuning project: you compare a tuned model against a baseline, and the tuned model’s "},{"text":"training data","i":true},{"text":" overlaps the evaluation while the baseline’s does not. The comparison is structurally unfair in the tuned model’s favour, and the gap you measure is partly recall."}]},{"t":"h2","id":"the-defences","runs":[{"text":"The defences"}]},{"t":"ol","start":1,"items":[[{"text":"Build the evaluation set from data the training set cannot contain","b":true},{"text":": a later time period, a held-out group, or items written specifically for evaluation and never released into the training pipeline."}],[{"text":"Run the n-gram check","b":true},{"text":" across your split, and report the number in your evaluation write-up even when it is zero. A stated zero is information; an unstated one is an assumption."}],[{"text":"Do not quote public benchmarks","b":true},{"text":" as evidence that your fine-tune worked. They cannot be cleaned of base-model contamination, and they are not measuring your task anyway."}],[{"text":"Keep a locked set","b":true},{"text":" created after the training data’s cutoff date. Temporal separation is the one defence that also protects against the contamination you cannot inspect, because a benchmark that did not exist when the base model was trained cannot be in its corpus."}]]},{"t":"h2","id":"the-summary","runs":[{"text":"The summary"}]},{"t":"p","runs":[{"text":"You can rule out contamination between your training set and your evaluation set. You cannot rule it out between the base model’s pretraining and any public benchmark. Say which of the two you have checked, and do not let a reader assume you checked both. That sentence, written into an evaluation report, is worth more than an additional decimal place on any of its numbers."}]}],"check":{"question":[{"text":"A tuned model scores markedly higher on a public multiple-choice benchmark when the answer options are in their canonical order than when the same options are shuffled. What does this indicate?"}],"options":[{"text":[{"text":"The model has seen those questions in that exact arrangement, so its score reflects recall."}],"correct":true,"tempting":null},{"text":[{"text":"The fine-tuning data taught a preference for the benchmark’s formatting conventions, which shuffling disrupts."}],"correct":false,"tempting":[{"text":"Formatting is unchanged by shuffling the options; only their arrangement differs, so a formatting preference cannot explain the gap."}]},{"text":[{"text":"Position bias in the model, which prefers the first listed option regardless of content."}],"correct":false,"tempting":[{"text":"Names a real bias, but position bias would affect canonical and shuffled orderings alike rather than distinguishing them."}]},{"text":[{"text":"The shuffled version is genuinely harder, because reordering breaks the logical progression the question author intended."}],"correct":false,"tempting":[{"text":"Assumes option order carries meaning the model exploits legitimately, which would not produce a consistent gap across unrelated questions."}]}],"order":[3,0,1,2],"why":null},"prev":{"slug":"judging-a-model-you-distilled","title":"Judging a model you distilled from the judge","minutes":9},"next":{"slug":"safety-regression-after-tuning","title":"Safety regression, including from benign data","minutes":9}}}