{"course":{"slug":"evaluating-ai","title":"Knowing If It Works","url":"https://addaly.com/learn/evaluating-ai","level":"intermediate","track":"build"},"lesson":{"slug":"measuring-text","title":"What to measure when the output is prose","url":"https://addaly.com/learn/evaluating-ai/measuring-text","position":11,"of":76,"minutes":8,"answer":"Decompose quality into checks that are separately true or false; one blended score hides where it broke.","keyIdea":"Decompose quality into checks that are separately true or false; one blended score hides where it broke."},"body":{"format":"markdown","text":"## Stop scoring \"quality\"\n\nThe instinct is to rate each output 1 to 5 for quality. Resist it. A 5-point score means something different to each labeller, something different to the same labeller on a Friday, and nothing at all to the engineer who has to act on it. \"We went from 3.4 to 3.6\" points at no change you can make.\n\nSplit the thing you care about into checks that are separately true or false. Five binary checks beat one 5-point score, every time, for the same labelling effort.\n\n## Three families of check, best first\n\n**Extract and compare exactly.** Most \"open-ended\" tasks have a right answer buried in the prose. A summary of an invoice contains an amount. A reply about a refund contains a policy window. A support answer names a product that either exists or does not. Pull that value out and compare it with `==`. This feels crude and it is the strongest signal you will get: it does not drift, it does not need a judge, and it is impossible to argue with.\n\n**Rubric checks, binary, written as questions.** \"Does it reply in the language the customer wrote in?\" \"Does it avoid promising a refund?\" \"Is it under 120 words?\" \"Does every link it gives resolve?\" Each is yes or no, each is cheap to check, and when the score drops you know which one moved.\n\n**Behavioural checks: did the next thing work.** Did the JSON parse. Did the SQL run without error. Did the tool call have valid arguments. Did the customer's next message say \"thank you\" rather than \"no, I meant the other order\". These cost nothing to collect and they are the closest thing to ground truth you will ever have.\n\n```python\ndef grade(item, output):\n    return {\n        \"parses\":        is_json(output),\n        \"amount_exact\":  extract_amount(output) == item[\"expected\"][\"amount\"],\n        \"currency_exact\": extract_currency(output) == item[\"expected\"][\"currency\"],\n        \"same_language\": detect_lang(output) == detect_lang(item[\"input\"]),\n        \"no_refund_promise\": not PROMISE_RE.search(output),\n    }\n    # store the dict, per item. Never collapse to one number here.\n```\n\nKeep every check separately for every item. Report the pass rate of each check and the pass rate within each tag. An overall number is for the meeting; the per-check table is for the work.\n\n## The metrics that will waste your week\n\nBLEU, ROUGE, and embedding-similarity-to-a-reference are widely used and mostly wrong for this. They were built for machine translation and for summarisation scored against reference corpora, where many valid outputs share vocabulary with the reference. Your task is not that. A support reply can share almost no words with your reference answer and be perfect; it can share most of them and be confidently wrong about the refund window.\n\nWhen a similarity score moves, the honest reading is \"the wording moved\", not \"the quality moved\". Occasionally that is what you want: detecting that a prompt change made outputs drift stylistically is a real use. Just do not report it as accuracy.\n\n## A note on the ones you cannot check\n\nSome things resist binary checks: tone, whether an explanation is at the right level, whether a piece of writing is any good. You have two honest options. Have humans rate them, in pairwise comparisons rather than absolute scores, on a small sample. Or use a model as a judge, which works, and misleads in specific ways that the next lesson is entirely about.\n\nWhat you should not do is invent a number for them and put it on a dashboard, because a number on a dashboard gets optimised, and a fake number gets faithfully optimised into a worse product."},"check":{"question":"A team measures its summarisation feature by cosine similarity between the generated summary and a human-written reference. After a prompt change the mean similarity rises from 0.71 to 0.79. What has been established?","options":[{"text":"That the new summaries use wording closer to the references; whether they are more accurate has not been tested.","correct":true},{"text":"That the summaries improved meaningfully: eight points on a zero-to-one scale is a large move.","correct":false,"tempting":"Treats a large movement in a number as evidence of quality, without asking what the number responds to."},{"text":"Nothing, because embedding similarity between texts is essentially random noise.","correct":false,"tempting":"Over-corrects into dismissing the metric entirely, when it does carry real signal about wording, just not about correctness."},{"text":"A valid improvement that now needs a significance test across the 100 items before shipping.","correct":false,"tempting":"Assumes the weakness in a metric is always statistical, when here the metric is measuring the wrong property precisely."}]},"prev":{"slug":"error-bars-you-can-compute","title":"How wrong your number probably is","url":"https://addaly.com/learn/evaluating-ai/error-bars-you-can-compute"},"next":{"slug":"the-confusion-matrix","title":"Precision and recall, worked on real numbers","url":"https://addaly.com/learn/evaluating-ai/the-confusion-matrix"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What to measure when the output is prose', in Knowing If It Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/evaluating-ai/measuring-text (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What to measure when the output is prose","courseTitle":"Knowing If It Works","module":{"index":2,"slug":"metrics-that-mean-something","title":"Metrics that mean something"},"answer":null,"keyIdea":"Decompose quality into checks that are separately true or false; one blended score hides where it broke.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"stop-scoring-quality","runs":[{"text":"Stop scoring “quality”"}]},{"t":"p","runs":[{"text":"The instinct is to rate each output 1 to 5 for quality. Resist it. A 5-point score means something different to each labeller, something different to the same labeller on a Friday, and nothing at all to the engineer who has to act on it. “We went from 3.4 to 3.6” points at no change you can make."}]},{"t":"p","runs":[{"text":"Split the thing you care about into checks that are separately true or false. Five binary checks beat one 5-point score, every time, for the same labelling effort."}]},{"t":"h2","id":"three-families-of-check-best-first","runs":[{"text":"Three families of check, best first"}]},{"t":"p","runs":[{"text":"Extract and compare exactly.","b":true},{"text":" Most “open-ended” tasks have a right answer buried in the prose. A summary of an invoice contains an amount. A reply about a refund contains a policy window. A support answer names a product that either exists or does not. Pull that value out and compare it with "},{"text":"==","code":true},{"text":". This feels crude and it is the strongest signal you will get: it does not drift, it does not need a judge, and it is impossible to argue with."}]},{"t":"p","runs":[{"text":"Rubric checks, binary, written as questions.","b":true},{"text":" “Does it reply in the language the customer wrote in?” “Does it avoid promising a refund?” “Is it under 120 words?” “Does every link it gives resolve?” Each is yes or no, each is cheap to check, and when the score drops you know which one moved."}]},{"t":"p","runs":[{"text":"Behavioural checks: did the next thing work.","b":true},{"text":" Did the JSON parse. Did the SQL run without error. Did the tool call have valid arguments. Did the customer’s next message say “thank you” rather than “no, I meant the other order”. These cost nothing to collect and they are the closest thing to ground truth you will ever have."}]},{"t":"code","lang":"python","label":"Python","text":"def grade(item, output):\n    return {\n        \"parses\":        is_json(output),\n        \"amount_exact\":  extract_amount(output) == item[\"expected\"][\"amount\"],\n        \"currency_exact\": extract_currency(output) == item[\"expected\"][\"currency\"],\n        \"same_language\": detect_lang(output) == detect_lang(item[\"input\"]),\n        \"no_refund_promise\": not PROMISE_RE.search(output),\n    }\n    # store the dict, per item. Never collapse to one number here."},{"t":"p","runs":[{"text":"Keep every check separately for every item. Report the pass rate of each check and the pass rate within each tag. An overall number is for the meeting; the per-check table is for the work."}]},{"t":"h2","id":"the-metrics-that-will-waste-your-week","runs":[{"text":"The metrics that will waste your week"}]},{"t":"p","runs":[{"text":"BLEU, ROUGE, and embedding-similarity-to-a-reference are widely used and mostly wrong for this. They were built for machine translation and for summarisation scored against reference corpora, where many valid outputs share vocabulary with the reference. Your task is not that. A support reply can share almost no words with your reference answer and be perfect; it can share most of them and be confidently wrong about the refund window."}]},{"t":"p","runs":[{"text":"When a similarity score moves, the honest reading is “the wording moved”, not “the quality moved”. Occasionally that is what you want: detecting that a prompt change made outputs drift stylistically is a real use. Just do not report it as accuracy."}]},{"t":"h2","id":"a-note-on-the-ones-you-cannot-check","runs":[{"text":"A note on the ones you cannot check"}]},{"t":"p","runs":[{"text":"Some things resist binary checks: tone, whether an explanation is at the right level, whether a piece of writing is any good. You have two honest options. Have humans rate them, in pairwise comparisons rather than absolute scores, on a small sample. Or use a model as a judge, which works, and misleads in specific ways that the next lesson is entirely about."}]},{"t":"p","runs":[{"text":"What you should not do is invent a number for them and put it on a dashboard, because a number on a dashboard gets optimised, and a fake number gets faithfully optimised into a worse product."}]}],"check":{"question":[{"text":"A team measures its summarisation feature by cosine similarity between the generated summary and a human-written reference. After a prompt change the mean similarity rises from 0.71 to 0.79. What has been established?"}],"options":[{"text":[{"text":"That the new summaries use wording closer to the references; whether they are more accurate has not been tested."}],"correct":true,"tempting":null},{"text":[{"text":"That the summaries improved meaningfully: eight points on a zero-to-one scale is a large move."}],"correct":false,"tempting":[{"text":"Treats a large movement in a number as evidence of quality, without asking what the number responds to."}]},{"text":[{"text":"Nothing, because embedding similarity between texts is essentially random noise."}],"correct":false,"tempting":[{"text":"Over-corrects into dismissing the metric entirely, when it does carry real signal about wording, just not about correctness."}]},{"text":[{"text":"A valid improvement that now needs a significance test across the 100 items before shipping."}],"correct":false,"tempting":[{"text":"Assumes the weakness in a metric is always statistical, when here the metric is measuring the wrong property precisely."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"error-bars-you-can-compute","title":"How wrong your number probably is","minutes":9},"next":{"slug":"the-confusion-matrix","title":"Precision and recall, worked on real numbers","minutes":9}}}