{"course":{"slug":"ai-and-you","title":"AI, Safety and What Goes Wrong","url":"https://addaly.com/learn/ai-and-you","level":"beginner","track":"start"},"lesson":{"slug":"the-benchmark-and-the-world","title":"What a benchmark score leaves out","url":"https://addaly.com/learn/ai-and-you/the-benchmark-and-the-world","position":4,"of":73,"minutes":3,"answer":"A benchmark measures a task under stated conditions. A matched local pilot adds evidence, with uncertainty.","keyIdea":"A benchmark measures a task under stated conditions. A matched local pilot adds evidence, with uncertainty."},"body":{"format":"markdown","text":"## Ninetieth percentile, against whom?\n\nIn its 2023 [GPT-4 technical report](https://arxiv.org/abs/2303.08774), OpenAI described a simulated US bar exam result near the top 10% of test takers. A percentile is a comparison. It tells you where the score sits in a reference group; it is not the share of questions answered correctly.\n\n[Eric Martínez's re-analysis](https://doi.org/10.1007/s10506-024-09396-9) found that approximate conversions from a February Illinois comparison, which included many repeat takers, could put the reported score near the 90th percentile. The July comparison gave an estimate around the 68th. Comparing with first-time July test takers gave around the 62nd. The paper also reports lower estimates when comparing only with passers, and identifies questions about how the essays were graded. The comparison and the scoring method belong beside the headline.\n\n> Figure: The same reported score against three groups. Bars (approx. percentile): February Illinois comparison 90; July comparison, overall 68; July comparison, first-time takers 62. Approximate percentile estimates from Martínez (2024), section 3.2.1. These show a relative position within each comparison group, not the percentage of questions answered correctly.\n\n## What a headline may omit\n\n**Some items may already be familiar.** In [the GSM1k study](https://proceedings.neurips.cc/paper_files/paper/2024/file/53384f2090c6a5cac952c598fd67992f-Paper-Datasets_and_Benchmarks_Track.pdf), researchers wrote new grade-school maths problems to resemble an earlier benchmark. Some tested models scored lower on the new set. A gap is a clue to examine alongside difficulty, prompts and other evidence. It cannot establish how a particular example entered a model's training.\n\n**The run may use help your product won't have.** Check which model version was tested, whether the prompt gave examples, which tools were allowed, whether multiple attempts were made and how the answer was scored.\n\n**The benchmark and your work may be different.** A multiple-choice question can test something useful, while missing incomplete inputs, language differences or the consequences of an error in a classroom.\n\n## Run a fair pilot\n\nThis is a separate exercise from the chapter film's room check. Here you compare two assistants on 20 fictional school notices with dates. Both assistants see the same source page and prompt for each notice. Decide what counts as a pass before looking at their answers: the date must match the source, and any missing date must be flagged instead of guessed.\n\n| Result | Notices |\n| --- | ---: |\n| Both pass | 12 |\n| A alone passes | 2 |\n| B alone passes | 5 |\n| Neither passes | 1 |\n\nA passes 14 of 20. B passes 17 of 20. That is a useful reason to inspect the seven cases where they differ. Check whether the misses cluster in a language, a type of notice or the most consequential situation. Twenty notices can reveal a problem; they cannot rule out rare failures. Use a fresh, larger set of examples before relying on the result for a more serious decision. [NIST's AI Risk Management Framework](https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/) recommends test sets representative of expected conditions and documenting the method.\n\nAsk the same questions of your own pilot and of a public benchmark: what was tested, under what conditions, against what comparison, and how much uncertainty remains?"},"check":{"question":"B passes 17 of 20 school notices and A passes 14. What can you conclude?","options":[{"text":"B is guaranteed to be better on all future school notices.","correct":false,"tempting":"A small sample cannot establish a guarantee."},{"text":"B did better on this pilot; inspect the disagreements and test a fresh set before relying on it.","correct":true},{"text":"A is better because its public benchmark score is higher.","correct":false,"tempting":"The public benchmark may measure a different task."},{"text":"The models are equal because the pilot is small.","correct":false,"tempting":"The observed counts differ. The size of the underlying difference is uncertain."}]},"prev":{"slug":"training-data-is-a-choice","title":"Where the data came from","url":"https://addaly.com/learn/ai-and-you/training-data-is-a-choice"},"next":{"slug":"when-the-failure-is-invisible","title":"Failure without an error message","url":"https://addaly.com/learn/ai-and-you/when-the-failure-is-invisible"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Source and assessment review continues. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What a benchmark score leaves out', in AI, Safety and What Goes Wrong. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-and-you/the-benchmark-and-the-world (Accessed: 21 November 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What a benchmark score leaves out","courseTitle":"AI, Safety and What Goes Wrong","module":{"index":1,"slug":"how-things-fail","title":"How these systems fail"},"answer":null,"keyIdea":"A benchmark measures a task under stated conditions. A matched local pilot adds evidence, with uncertainty.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","title":"The same reported score against three groups","caption":"Approximate percentile estimates from Martínez (2024), section 3.2.1. These show a relative position within each comparison group, not the percentage of questions answered correctly.","unit":"approx. percentile","series":[{"label":"February Illinois comparison","value":90},{"label":"July comparison, overall","value":68},{"label":"July comparison, first-time takers","value":62}]},"alt":"Figure: The same reported score against three groups. Bars (approx. percentile): February Illinois comparison 90; July comparison, overall 68; July comparison, first-time takers 62. Approximate percentile estimates from Martínez (2024), section 3.2.1. These show a relative position within each comparison group, not the percentage of questions answered correctly."}},"blocks":[{"t":"h2","id":"ninetieth-percentile-against-whom","runs":[{"text":"Ninetieth percentile, against whom?"}]},{"t":"p","runs":[{"text":"In its 2023 "},{"text":"GPT-4 technical report","href":"https://arxiv.org/abs/2303.08774"},{"text":", OpenAI described a simulated US bar exam result near the top 10% of test takers. A percentile is a comparison. It tells you where the score sits in a reference group; it is not the share of questions answered correctly."}]},{"t":"p","runs":[{"text":"Eric Martínez’s re-analysis","href":"https://doi.org/10.1007/s10506-024-09396-9"},{"text":" found that approximate conversions from a February Illinois comparison, which included many repeat takers, could put the reported score near the 90th percentile. The July comparison gave an estimate around the 68th. Comparing with first-time July test takers gave around the 62nd. The paper also reports lower estimates when comparing only with passers, and identifies questions about how the essays were graded. The comparison and the scoring method belong beside the headline."}]},{"t":"quick","prompt":"Place a bet: against lawyers who passed the exam, where did the same score land?","options":["About the 90th percentile","About the 70th","About the 48th"],"answer":2,"reveal":"About the 48th. The figure shows every comparison."},{"t":"h2","id":"what-a-headline-may-omit","runs":[{"text":"What a headline may omit"}]},{"t":"p","runs":[{"text":"Some items may already be familiar.","b":true},{"text":" In "},{"text":"the GSM1k study","href":"https://proceedings.neurips.cc/paper_files/paper/2024/file/53384f2090c6a5cac952c598fd67992f-Paper-Datasets_and_Benchmarks_Track.pdf"},{"text":", researchers wrote new grade-school maths problems to resemble an earlier benchmark. Some tested models scored lower on the new set. A gap is a clue to examine alongside difficulty, prompts and other evidence. It cannot establish how a particular example entered a model’s training."}]},{"t":"p","runs":[{"text":"The run may use help your product won’t have.","b":true},{"text":" Check which model version was tested, whether the prompt gave examples, which tools were allowed, whether multiple attempts were made and how the answer was scored."}]},{"t":"p","runs":[{"text":"The benchmark and your work may be different.","b":true},{"text":" A multiple-choice question can test something useful, while missing incomplete inputs, language differences or the consequences of an error in a classroom."}]},{"t":"h2","id":"run-a-fair-pilot","runs":[{"text":"Run a fair pilot"}]},{"t":"p","runs":[{"text":"This is a separate exercise from the chapter film’s room check. Here you compare two assistants on 20 fictional school notices with dates. Both assistants see the same source page and prompt for each notice. Decide what counts as a pass before looking at their answers: the date must match the source, and any missing date must be flagged instead of guessed."}]},{"t":"table","head":[[{"text":"Result"}],[{"text":"Notices"}]],"align":["left","right"],"rows":[[[{"text":"Both pass"}],[{"text":"12"}]],[[{"text":"A alone passes"}],[{"text":"2"}]],[[{"text":"B alone passes"}],[{"text":"5"}]],[[{"text":"Neither passes"}],[{"text":"1"}]]]},{"t":"p","runs":[{"text":"A passes 14 of 20. B passes 17 of 20. That is a useful reason to inspect the seven cases where they differ. Check whether the misses cluster in a language, a type of notice or the most consequential situation. Twenty notices can reveal a problem; they cannot rule out rare failures. Use a fresh, larger set of examples before relying on the result for a more serious decision. "},{"text":"NIST’s AI Risk Management Framework","href":"https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/"},{"text":" recommends test sets representative of expected conditions and documenting the method."}]},{"t":"p","runs":[{"text":"Ask the same questions of your own pilot and of a public benchmark: what was tested, under what conditions, against what comparison, and how much uncertainty remains?"}]}],"check":{"question":[{"text":"B passes 17 of 20 school notices and A passes 14. What can you conclude?"}],"options":[{"text":[{"text":"B is guaranteed to be better on all future school notices."}],"correct":false,"tempting":[{"text":"A small sample cannot establish a guarantee."}]},{"text":[{"text":"B did better on this pilot; inspect the disagreements and test a fresh set before relying on it."}],"correct":true,"tempting":null},{"text":[{"text":"A is better because its public benchmark score is higher."}],"correct":false,"tempting":[{"text":"The public benchmark may measure a different task."}]},{"text":[{"text":"The models are equal because the pilot is small."}],"correct":false,"tempting":[{"text":"The observed counts differ. The size of the underlying difference is uncertain."}]}],"order":[0,1,2,3],"why":null},"prev":{"slug":"training-data-is-a-choice","title":"Where the data came from","minutes":3},"next":{"slug":"when-the-failure-is-invisible","title":"Failure without an error message","minutes":3}}}