Two sets, and the ways they get contaminated

Why one set is not enough

You will look at your evaluation set constantly. You will read the failures, change the prompt to fix them, re-run, read the new failures, change it again. After twenty rounds of that, the score on that set is no longer an estimate of how the system behaves on new inputs. It is a record of how well you fitted a hundred specific examples, using your own brain as the optimiser.

This is overfitting, and it does not require a training loop. It happens to prompts, to retrieval configurations, to threshold choices, to anything you tune while watching a number.

The fix is two sets, with different rules:

• The development set. Look at it as often as you like. Read every failure. Tune against it. This is where the work happens.
• The test set. Do not read the items. Run it when a decision is being made (before a release, before telling somebody a number) and then leave it alone. Once a month is generous. Once a quarter is common.

Split them before you start, by a rule that cannot drift: a hash of the item id, int(md5(id)[:8], 16) % 10 < 2 for a 20% test set. Written that way, adding new items later keeps the split stable and nobody has to remember anything.

If the test number is much worse than the dev number, you have not been improving the system. You have been improving your fit to the dev set. That gap is the most useful diagnostic in this course and it is invisible with one set.

Near-duplicates are the commonest leak

Suppose your data comes from support tickets. “How do I reset my password” appears 340 times in slightly different wording. Sample randomly into two sets and near-identical items land on both sides. Your test set is now measuring the same question your dev set already taught you to answer, and it will look wonderful.

One team building an FAQ router reported 94% on a held-out set and 61% in production. Thirty of their hundred held-out items were paraphrases of items in the training prompt’s few-shot examples. The set was not testing generalisation at all.

Dedupe before splitting. A cheap and surprisingly effective pass: normalise (lowercase, strip punctuation, collapse whitespace), then drop exact hash matches; then embed everything with a small local model such as all-MiniLM-L6-v2 through sentence-transformers (free, runs on a laptop CPU) and drop anything with cosine similarity above about 0.95 to an item already kept. Look at twenty of the pairs you dropped before trusting the threshold.

Split by group, not by row

Rows are usually not independent. Twelve tickets from the same customer share vocabulary, product, and often the same underlying bug. Ten pages from the same document share formatting. Put some in dev and some in test and information crosses the wall.

So split by the unit that shares information: customer, document, seller, conversation, patient. All rows for one group go entirely to one side. GroupKFold in scikit-learn does this in one line, and doing it by hand in SQL is a GROUP BY and a hash.

Split by time when time matters

If what you are building will run on next month’s data, evaluating on a random sample of last year’s is optimistic. Language, product names, scam patterns and customer questions all move. Take the last four weeks as the test set and everything earlier as dev. The number will be lower than a random split gives you. The lower number is the honest one.

This also catches an error that random splits hide entirely: features computed with information that would not exist at prediction time. If your input includes a field the system only learns after the event you are predicting, a random split rewards it and a time split exposes it.

Benchmark contamination

There is a version of this problem you cannot fix by splitting, because the leak happened before you arrived. Public evaluation sets (the ones a model’s card quotes) are on the open web, and models are trained on the open web. A model may have seen the questions and the answers.

The signs are recognisable: near-perfect scores on an old public benchmark alongside ordinary performance on a freshly written variant of the same task; sensitivity to trivial rewording that should not matter; occasionally, verbatim recall of a question’s phrasing. Some benchmark authors embed canary strings, unique identifiers that should never appear in training data, so contamination can be detected after the fact. It usually is not checked.

The practical consequence is narrow and important: your own set, made from your own data, after your model’s training cutoff, is the only set you can be confident is uncontaminated. That is a large part of why lesson two asks you to build one by hand.

The rule that keeps it working

Write the rules down where the set lives: which split is which, who may look at what, and how often the test set is run. Then treat a test-set peek the way you would treat a schema migration: not forbidden, but noticed, logged and deliberate.

The first time somebody quietly tunes against the test set to make a release, the set stops being evidence, and nothing in the output tells you that it has happened.