{"course":{"slug":"evaluating-ai","title":"Knowing If It Works","url":"https://addaly.com/learn/evaluating-ai","level":"intermediate","track":"build"},"lesson":{"slug":"simpsons-paradox","title":"Better in every group and worse overall","url":"https://addaly.com/learn/evaluating-ai/simpsons-paradox","position":29,"of":76,"minutes":8,"answer":"An overall score is a weighted average, so a change in the mix of hard and easy items can reverse it while every group improves: fix the weights before comparing two numbers.","keyIdea":"An overall score is a weighted average, so a change in the mix of hard and easy items can reverse it while every group improves: fix the weights before comparing two numbers."},"body":{"format":"markdown","text":"## The reversal, in numbers\n\nTwo models, measured on Hindi and English requests.\n\nThe old model was measured on 500 Hindi items and 500 English items: 68% on Hindi, 89% on English. Overall, 78.5%.\n\nThe new model was measured on 800 Hindi and 200 English: 72% on Hindi, 91% on English. Overall:\n\n```\n0.8 * 72 + 0.2 * 91 = 57.6 + 18.2 = 75.8%\n```\n\nThe new model is better in Hindi, better in English, and 2.7 points worse overall. Nobody made an arithmetic error. The two overall numbers are weighted averages with different weights, and Hindi is the harder half.\n\n## Where the different weights come from\n\nThis is not a curiosity. Every one of these produces it:\n\n- **The set was refreshed between measurements.** Traffic grew in one market, so the new sample contains more of it.\n- **Assignment correlates with a segment.** The new experience shipped to the app first, and app users differ from web users.\n- **A router sends hard cases to the new system.** Any escalation rule guarantees the new system is measured on a harder mix.\n- **A canary is regional.** The 5% rollout went to one data centre, which serves one part of the world.\n- **Volume itself moved during the experiment.** A campaign, an outage, a public holiday.\n\nThe last one is the reason a production metric can move several points with no change to the model at all. That is a mix shift, and the first question when a dashboard drops should be \"did the mix move\" rather than \"what broke\".\n\n## The fix is to fix the weights\n\nReport per slice, always. Then, if a single number is required, compute it against a fixed reference population: direct standardisation:\n\n```python\nref = {\"hi\": 0.55, \"en\": 0.45}                        # frozen traffic mix\noverall = sum(ref[s] * rate[s] for s in ref)\n```\n\nBoth systems are now scored on the same imaginary population, so the comparison is valid. Write the reference weights into the report and version them; when the real mix moves far enough that the reference is fiction, update it deliberately and re-baseline everything, rather than letting it drift silently.\n\nThe same trick answers the monitoring question. Track the raw metric and the mix-standardised metric side by side. When the raw one moves and the standardised one does not, your users changed rather than your system.\n\nThere is one more guard worth building into the harness: refuse to print a single overall number when the slice proportions of the two runs differ by more than a few points. A comparison whose weights moved is not a comparison, and a harness that says so is more useful than one that quietly averages. Ten lines of code, and it catches the case where somebody re-sampled the eval set between two runs and did not mention it.\n\n## The reverse trap\n\nKnowing about this paradox creates a second temptation: slice until some grouping shows the story you want. Any set can be partitioned to produce almost any conclusion, and choosing the partition after seeing the outcomes is exactly the multiplicity problem from earlier in this block.\n\nTwo disciplines prevent it. Fix the slice definitions before the comparison, in the same document as the primary metric. And require a mechanism: a slicing variable belongs in the analysis if you can say why it would affect quality (script, input length, retrieval coverage) not because it separated the results.\n\n## The part statistics cannot settle\n\nThere is no test that tells you whether the grouped view or the overall view is the correct one. The choice depends on a claim about what causes what, and that claim comes from you.\n\nIf language causes difficulty and the mix changed, the per-language numbers are the truth and the overall reversal is an artefact. If instead your change itself caused the mix to shift (a new interface that attracts different users) then the overall number reflects a real consequence of the change, and quoting only the per-group improvement hides it.\n\nSo state the assumption out loud in the report: \"we treat language mix as external to this change, and standardise to the March traffic mix\". A reader who disagrees can then say so precisely, which is the most that any honest analysis of this kind can offer."},"check":{"question":"A production quality metric drops from 86% to 82% overnight, while every language slice holds steady or improves. What should be checked first?","options":[{"text":"Whether the model version changed, since a silent provider-side update is the usual cause of an overnight drop.","correct":false,"tempting":"A real hazard covered later in the course, but it would be expected to move the slice-level numbers too, and here every slice held."},{"text":"Whether the traffic mix shifted towards a slice with a lower pass rate, since the overall figure is a weighted average of the slices.","correct":true},{"text":"Whether the evaluation set was accidentally reordered, causing items to be scored against the wrong references.","correct":false,"tempting":"A harness bug of this kind would corrupt the slice numbers as well, so it cannot explain slices that held steady."},{"text":"Whether one slice grew large enough to be significant, making a previously hidden regression visible.","correct":false,"tempting":"Misreads growth in a slice as newly revealing a regression; the slice rates themselves did not fall, which is what a hidden regression would require."}]},"prev":{"slug":"variance-reduction","title":"Getting more signal from the items you already have","url":"https://addaly.com/learn/evaluating-ai/variance-reduction"},"next":{"slug":"evaluating-summaries","title":"Summaries: what was invented, and what was left out","url":"https://addaly.com/learn/evaluating-ai/evaluating-summaries"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Better in every group and worse overall', in Knowing If It Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/evaluating-ai/simpsons-paradox (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Better in every group and worse overall","courseTitle":"Knowing If It Works","module":{"index":3,"slug":"deciding-if-it-is-real","title":"Deciding whether a difference is real"},"answer":null,"keyIdea":"An overall score is a weighted average, so a change in the mix of hard and easy items can reverse it while every group improves: fix the weights before comparing two numbers.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-reversal-in-numbers","runs":[{"text":"The reversal, in numbers"}]},{"t":"p","runs":[{"text":"Two models, measured on Hindi and English requests."}]},{"t":"p","runs":[{"text":"The old model was measured on 500 Hindi items and 500 English items: 68% on Hindi, 89% on English. Overall, 78.5%."}]},{"t":"p","runs":[{"text":"The new model was measured on 800 Hindi and 200 English: 72% on Hindi, 91% on English. Overall:"}]},{"t":"code","lang":"","label":"Text","text":"0.8 * 72 + 0.2 * 91 = 57.6 + 18.2 = 75.8%"},{"t":"p","runs":[{"text":"The new model is better in Hindi, better in English, and 2.7 points worse overall. Nobody made an arithmetic error. The two overall numbers are weighted averages with different weights, and Hindi is the harder half."}]},{"t":"h2","id":"where-the-different-weights-come-from","runs":[{"text":"Where the different weights come from"}]},{"t":"p","runs":[{"text":"This is not a curiosity. Every one of these produces it:"}]},{"t":"ul","items":[[{"text":"The set was refreshed between measurements.","b":true},{"text":" Traffic grew in one market, so the new sample contains more of it."}],[{"text":"Assignment correlates with a segment.","b":true},{"text":" The new experience shipped to the app first, and app users differ from web users."}],[{"text":"A router sends hard cases to the new system.","b":true},{"text":" Any escalation rule guarantees the new system is measured on a harder mix."}],[{"text":"A canary is regional.","b":true},{"text":" The 5% rollout went to one data centre, which serves one part of the world."}],[{"text":"Volume itself moved during the experiment.","b":true},{"text":" A campaign, an outage, a public holiday."}]]},{"t":"p","runs":[{"text":"The last one is the reason a production metric can move several points with no change to the model at all. That is a mix shift, and the first question when a dashboard drops should be “did the mix move” rather than “what broke”."}]},{"t":"h2","id":"the-fix-is-to-fix-the-weights","runs":[{"text":"The fix is to fix the weights"}]},{"t":"p","runs":[{"text":"Report per slice, always. Then, if a single number is required, compute it against a fixed reference population: direct standardisation:"}]},{"t":"code","lang":"python","label":"Python","text":"ref = {\"hi\": 0.55, \"en\": 0.45}                        # frozen traffic mix\noverall = sum(ref[s] * rate[s] for s in ref)"},{"t":"p","runs":[{"text":"Both systems are now scored on the same imaginary population, so the comparison is valid. Write the reference weights into the report and version them; when the real mix moves far enough that the reference is fiction, update it deliberately and re-baseline everything, rather than letting it drift silently."}]},{"t":"p","runs":[{"text":"The same trick answers the monitoring question. Track the raw metric and the mix-standardised metric side by side. When the raw one moves and the standardised one does not, your users changed rather than your system."}]},{"t":"p","runs":[{"text":"There is one more guard worth building into the harness: refuse to print a single overall number when the slice proportions of the two runs differ by more than a few points. A comparison whose weights moved is not a comparison, and a harness that says so is more useful than one that quietly averages. Ten lines of code, and it catches the case where somebody re-sampled the eval set between two runs and did not mention it."}]},{"t":"h2","id":"the-reverse-trap","runs":[{"text":"The reverse trap"}]},{"t":"p","runs":[{"text":"Knowing about this paradox creates a second temptation: slice until some grouping shows the story you want. Any set can be partitioned to produce almost any conclusion, and choosing the partition after seeing the outcomes is exactly the multiplicity problem from earlier in this block."}]},{"t":"p","runs":[{"text":"Two disciplines prevent it. Fix the slice definitions before the comparison, in the same document as the primary metric. And require a mechanism: a slicing variable belongs in the analysis if you can say why it would affect quality (script, input length, retrieval coverage) not because it separated the results."}]},{"t":"h2","id":"the-part-statistics-cannot-settle","runs":[{"text":"The part statistics cannot settle"}]},{"t":"p","runs":[{"text":"There is no test that tells you whether the grouped view or the overall view is the correct one. The choice depends on a claim about what causes what, and that claim comes from you."}]},{"t":"p","runs":[{"text":"If language causes difficulty and the mix changed, the per-language numbers are the truth and the overall reversal is an artefact. If instead your change itself caused the mix to shift (a new interface that attracts different users) then the overall number reflects a real consequence of the change, and quoting only the per-group improvement hides it."}]},{"t":"p","runs":[{"text":"So state the assumption out loud in the report: “we treat language mix as external to this change, and standardise to the March traffic mix”. A reader who disagrees can then say so precisely, which is the most that any honest analysis of this kind can offer."}]}],"check":{"question":[{"text":"A production quality metric drops from 86% to 82% overnight, while every language slice holds steady or improves. What should be checked first?"}],"options":[{"text":[{"text":"Whether the model version changed, since a silent provider-side update is the usual cause of an overnight drop."}],"correct":false,"tempting":[{"text":"A real hazard covered later in the course, but it would be expected to move the slice-level numbers too, and here every slice held."}]},{"text":[{"text":"Whether the traffic mix shifted towards a slice with a lower pass rate, since the overall figure is a weighted average of the slices."}],"correct":true,"tempting":null},{"text":[{"text":"Whether the evaluation set was accidentally reordered, causing items to be scored against the wrong references."}],"correct":false,"tempting":[{"text":"A harness bug of this kind would corrupt the slice numbers as well, so it cannot explain slices that held steady."}]},{"text":[{"text":"Whether one slice grew large enough to be significant, making a previously hidden regression visible."}],"correct":false,"tempting":[{"text":"Misreads growth in a slice as newly revealing a regression; the slice rates themselves did not fall, which is what a hidden regression would require."}]}],"order":[2,0,1,3],"why":null},"prev":{"slug":"variance-reduction","title":"Getting more signal from the items you already have","minutes":9},"next":{"slug":"evaluating-summaries","title":"Summaries: what was invented, and what was left out","minutes":9}}}