{"course":{"slug":"evaluating-ai","title":"Knowing If It Works","url":"https://addaly.com/learn/evaluating-ai","level":"intermediate","track":"build"},"lesson":{"slug":"thresholds-and-the-cost-of-being-wrong","title":"Moving the threshold is a business decision","url":"https://addaly.com/learn/evaluating-ai/thresholds-and-the-cost-of-being-wrong","position":15,"of":76,"minutes":10,"answer":"The threshold is not a model parameter but a price you set on the two kinds of mistake, and a band with a human in the middle usually beats a single line.","keyIdea":"The threshold is not a model parameter but a price you set on the two kinds of mistake, and a band with a human in the middle usually beats a single line."},"body":{"format":"markdown","text":"## The model gives a score; you choose the line\n\nMost classifiers do not output yes or no. They output a number between 0 and 1, and somebody (usually by default, usually without noticing) decided that 0.5 means yes.\n\n0.5 is not a principled choice. It is the midpoint of a range, and the range is arbitrary. The threshold is where you convert a score into an action, and it should be set by what the two kinds of mistake cost you.\n\nTake the content filter from the previous lesson: 1,000 posts, 60 harmful. Sweep the threshold and the picture changes completely.\n\n- At **0.3**: flags 210 posts, catches 57 of 60. Recall 95%, precision 27%. The review queue has 153 innocent posts in it.\n- At **0.5**: flags 80, catches 48. Recall 80%, precision 60%.\n- At **0.8**: flags 38, catches 34. Recall 57%, precision 89%. Twenty-six harmful posts go through.\n\nSame model, same weights, three different products. Nobody retrained anything.\n\n## Put a number on each mistake\n\nYou can do better than arguing about it. Assign a cost to a false positive and a false negative, in any consistent unit, and pick the threshold that minimises total cost.\n\nSay a false positive costs you five minutes of a moderator's time, and a false negative costs the equivalent of two hours of harm, complaints and trust. That is a ratio of 24 to 1.\n\n```\ncost = FP * 5 + FN * 120     (in minutes)\n\nthreshold 0.3:  153*5 + 3*120  =  765 + 360  = 1125\nthreshold 0.5:   32*5 + 12*120 =  160 + 1440 = 1600\nthreshold 0.8:    4*5 + 26*120 =   20 + 3120 = 3140\n```\n\n> Figure: Total cost of mistakes at three thresholds. Bars (minutes of cost per 1,000 posts): 0.3: 153 false alarms, 3 misses 1125; 0.5: 32 false alarms, 12 misses 1600; 0.8: 4 false alarms, 26 misses 3140. At five minutes per false alarm and two hours per miss, the low threshold wins by a wide margin. Change the 24:1 ratio and the winner moves, which is the argument worth having.\n\nThe low threshold wins by a wide margin, and now the argument is about the 24:1 ratio, which is the argument you should have been having all along. People who will not agree on a threshold will often agree on a ratio, and the ratio is the honest form of the disagreement.\n\nThe numbers are made up. Making them up explicitly, in a document, is still much better than leaving them implicit in a default.\n\n## PR curves and the ROC trap\n\nSweeping the threshold and plotting the results gives you a curve, and there are two conventions.\n\nThe **ROC curve** plots true positive rate against false positive rate, and its area (AUC) is the number most commonly quoted. Under heavy imbalance it is misleading, because the false positive rate has 940 true negatives in its denominator. Going from 32 to 153 false positives moves that rate from 3.4% to 16%; the curve barely notices, while your review queue has grown fivefold.\n\nThe **precision–recall curve** puts precision on one axis, and precision has the flagged count in its denominator, so it feels every one of those false positives. For rare positives, read the PR curve. Report average precision rather than ROC-AUC.\n\nBoth are one line in scikit-learn (`precision_recall_curve`, `roc_curve`), free, and worth plotting once even if you never plot it again: seeing your own curve's shape tells you whether there is a threshold that gets you both, or whether the model is not separating the classes and no threshold will save you.\n\n## Scores are not probabilities\n\nA score of 0.9 does not mean \"right 90% of the time\" unless the model has been calibrated. Check it directly: bucket predictions into ten bins by score, and in each bin compare the mean predicted score to the observed fraction of positives. If the 0.9 bucket is right 62% of the time, your scores are inflated, which is normal for neural networks.\n\nFixing it is cheap: Platt scaling (a logistic regression on the scores) or isotonic regression, both in `sklearn.calibration`, both needing only a held-out set with labels. Calibrate before you compute expected cost, because the cost arithmetic above assumes the score means something.\n\nFor a large language model asked to rate its own confidence, none of this applies and calibration is worse than poor. Self-reported confidence clusters near 0.9 regardless of correctness. If you need a usable score from an LLM classifier, take the token log-probabilities of the label tokens where the provider exposes them, and calibrate those. Treat a number the model has typed out in prose as text, not as a probability.\n\n## Two thresholds are often better than one\n\nThe most useful shape in practice is not one line but two. Below 0.3, act automatically as negative. Above 0.85, act automatically as positive. Between, send it to a human.\n\nThis lets you tune the middle band to your actual review capacity: if the queue can absorb 200 items a day, widen or narrow the band until it does. You get high precision on both automatic paths and you spend your scarce human attention exactly where the model is unsure. It is also the design that degrades gracefully, because when volume spikes the band can be narrowed rather than the whole system switched off."},"check":{"question":"A team reports ROC-AUC of 0.94 for a fraud model where 0.8% of transactions are fraudulent, and proposes shipping at the default 0.5 threshold. What is the most substantive objection?","options":[{"text":"AUC cannot be compared across datasets, so the 0.94 is meaningless without a baseline.","correct":false,"tempting":"Raises a genuine caveat about cross-dataset comparison that does not bear on whether this threshold is right for this system."},{"text":"The model has not been retrained on recent data, so the score distribution is probably stale.","correct":false,"tempting":"Introduces a drift concern with no evidence for it, while the stated configuration already contains a concrete problem."},{"text":"0.94 is not high enough for a production fraud system; anything under 0.99 will generate too many alerts.","correct":false,"tempting":"Treats AUC as if it had an absolute pass mark, when what matters is where the operating point sits on the curve."},{"text":"At a 0.8% base rate, ROC-AUC hides the false-positive burden and 0.5 is an arbitrary operating point; they should read the precision–recall curve and pick a threshold from the cost of each mistake.","correct":true}]},"prev":{"slug":"reading-the-curve","title":"What a curve says, and what its one number leaves out","url":"https://addaly.com/learn/evaluating-ai/reading-the-curve"},"next":{"slug":"probability-calibration","title":"Making a 0.8 mean eighty per cent","url":"https://addaly.com/learn/evaluating-ai/probability-calibration"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Moving the threshold is a business decision', in Knowing If It Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/evaluating-ai/thresholds-and-the-cost-of-being-wrong (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Moving the threshold is a business decision","courseTitle":"Knowing If It Works","module":{"index":2,"slug":"metrics-that-mean-something","title":"Metrics that mean something"},"answer":null,"keyIdea":"The threshold is not a model parameter but a price you set on the two kinds of mistake, and a band with a human in the middle usually beats a single line.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"minutes of cost per 1,000 posts","title":"Total cost of mistakes at three thresholds","series":[{"label":"0.3: 153 false alarms, 3 misses","value":1125,"highlight":true},{"label":"0.5: 32 false alarms, 12 misses","value":1600},{"label":"0.8: 4 false alarms, 26 misses","value":3140}],"caption":"At five minutes per false alarm and two hours per miss, the low threshold wins by a wide margin. Change the 24:1 ratio and the winner moves, which is the argument worth having."},"alt":"Figure: Total cost of mistakes at three thresholds. Bars (minutes of cost per 1,000 posts): 0.3: 153 false alarms, 3 misses 1125; 0.5: 32 false alarms, 12 misses 1600; 0.8: 4 false alarms, 26 misses 3140. At five minutes per false alarm and two hours per miss, the low threshold wins by a wide margin. Change the 24:1 ratio and the winner moves, which is the argument worth having."}},"blocks":[{"t":"h2","id":"the-model-gives-a-score-you-choose-the-line","runs":[{"text":"The model gives a score; you choose the line"}]},{"t":"p","runs":[{"text":"Most classifiers do not output yes or no. They output a number between 0 and 1, and somebody (usually by default, usually without noticing) decided that 0.5 means yes."}]},{"t":"p","runs":[{"text":"0.5 is not a principled choice. It is the midpoint of a range, and the range is arbitrary. The threshold is where you convert a score into an action, and it should be set by what the two kinds of mistake cost you."}]},{"t":"p","runs":[{"text":"Take the content filter from the previous lesson: 1,000 posts, 60 harmful. Sweep the threshold and the picture changes completely."}]},{"t":"ul","items":[[{"text":"At "},{"text":"0.3","b":true},{"text":": flags 210 posts, catches 57 of 60. Recall 95%, precision 27%. The review queue has 153 innocent posts in it."}],[{"text":"At "},{"text":"0.5","b":true},{"text":": flags 80, catches 48. Recall 80%, precision 60%."}],[{"text":"At "},{"text":"0.8","b":true},{"text":": flags 38, catches 34. Recall 57%, precision 89%. Twenty-six harmful posts go through."}]]},{"t":"p","runs":[{"text":"Same model, same weights, three different products. Nobody retrained anything."}]},{"t":"h2","id":"put-a-number-on-each-mistake","runs":[{"text":"Put a number on each mistake"}]},{"t":"p","runs":[{"text":"You can do better than arguing about it. Assign a cost to a false positive and a false negative, in any consistent unit, and pick the threshold that minimises total cost."}]},{"t":"p","runs":[{"text":"Say a false positive costs you five minutes of a moderator’s time, and a false negative costs the equivalent of two hours of harm, complaints and trust. That is a ratio of 24 to 1."}]},{"t":"code","lang":"","label":"Text","text":"cost = FP * 5 + FN * 120     (in minutes)\n\nthreshold 0.3:  153*5 + 3*120  =  765 + 360  = 1125\nthreshold 0.5:   32*5 + 12*120 =  160 + 1440 = 1600\nthreshold 0.8:    4*5 + 26*120 =   20 + 3120 = 3140"},{"t":"p","runs":[{"text":"The low threshold wins by a wide margin, and now the argument is about the 24:1 ratio, which is the argument you should have been having all along. People who will not agree on a threshold will often agree on a ratio, and the ratio is the honest form of the disagreement."}]},{"t":"p","runs":[{"text":"The numbers are made up. Making them up explicitly, in a document, is still much better than leaving them implicit in a default."}]},{"t":"h2","id":"pr-curves-and-the-roc-trap","runs":[{"text":"PR curves and the ROC trap"}]},{"t":"p","runs":[{"text":"Sweeping the threshold and plotting the results gives you a curve, and there are two conventions."}]},{"t":"p","runs":[{"text":"The "},{"text":"ROC curve","b":true},{"text":" plots true positive rate against false positive rate, and its area (AUC) is the number most commonly quoted. Under heavy imbalance it is misleading, because the false positive rate has 940 true negatives in its denominator. Going from 32 to 153 false positives moves that rate from 3.4% to 16%; the curve barely notices, while your review queue has grown fivefold."}]},{"t":"p","runs":[{"text":"The "},{"text":"precision–recall curve","b":true},{"text":" puts precision on one axis, and precision has the flagged count in its denominator, so it feels every one of those false positives. For rare positives, read the PR curve. Report average precision rather than ROC-AUC."}]},{"t":"p","runs":[{"text":"Both are one line in scikit-learn ("},{"text":"precision_recall_curve","code":true},{"text":", "},{"text":"roc_curve","code":true},{"text":"), free, and worth plotting once even if you never plot it again: seeing your own curve’s shape tells you whether there is a threshold that gets you both, or whether the model is not separating the classes and no threshold will save you."}]},{"t":"h2","id":"scores-are-not-probabilities","runs":[{"text":"Scores are not probabilities"}]},{"t":"p","runs":[{"text":"A score of 0.9 does not mean “right 90% of the time” unless the model has been calibrated. Check it directly: bucket predictions into ten bins by score, and in each bin compare the mean predicted score to the observed fraction of positives. If the 0.9 bucket is right 62% of the time, your scores are inflated, which is normal for neural networks."}]},{"t":"p","runs":[{"text":"Fixing it is cheap: Platt scaling (a logistic regression on the scores) or isotonic regression, both in "},{"text":"sklearn.calibration","code":true},{"text":", both needing only a held-out set with labels. Calibrate before you compute expected cost, because the cost arithmetic above assumes the score means something."}]},{"t":"p","runs":[{"text":"For a large language model asked to rate its own confidence, none of this applies and calibration is worse than poor. Self-reported confidence clusters near 0.9 regardless of correctness. If you need a usable score from an LLM classifier, take the token log-probabilities of the label tokens where the provider exposes them, and calibrate those. Treat a number the model has typed out in prose as text, not as a probability."}]},{"t":"h2","id":"two-thresholds-are-often-better-than-one","runs":[{"text":"Two thresholds are often better than one"}]},{"t":"p","runs":[{"text":"The most useful shape in practice is not one line but two. Below 0.3, act automatically as negative. Above 0.85, act automatically as positive. Between, send it to a human."}]},{"t":"p","runs":[{"text":"This lets you tune the middle band to your actual review capacity: if the queue can absorb 200 items a day, widen or narrow the band until it does. You get high precision on both automatic paths and you spend your scarce human attention exactly where the model is unsure. It is also the design that degrades gracefully, because when volume spikes the band can be narrowed rather than the whole system switched off."}]}],"check":{"question":[{"text":"A team reports ROC-AUC of 0.94 for a fraud model where 0.8% of transactions are fraudulent, and proposes shipping at the default 0.5 threshold. What is the most substantive objection?"}],"options":[{"text":[{"text":"AUC cannot be compared across datasets, so the 0.94 is meaningless without a baseline."}],"correct":false,"tempting":[{"text":"Raises a genuine caveat about cross-dataset comparison that does not bear on whether this threshold is right for this system."}]},{"text":[{"text":"The model has not been retrained on recent data, so the score distribution is probably stale."}],"correct":false,"tempting":[{"text":"Introduces a drift concern with no evidence for it, while the stated configuration already contains a concrete problem."}]},{"text":[{"text":"0.94 is not high enough for a production fraud system; anything under 0.99 will generate too many alerts."}],"correct":false,"tempting":[{"text":"Treats AUC as if it had an absolute pass mark, when what matters is where the operating point sits on the curve."}]},{"text":[{"text":"At a 0.8% base rate, ROC-AUC hides the false-positive burden and 0.5 is an arbitrary operating point; they should read the precision–recall curve and pick a threshold from the cost of each mistake."}],"correct":true,"tempting":null}],"order":[1,2,3,0],"why":null},"prev":{"slug":"reading-the-curve","title":"What a curve says, and what its one number leaves out","minutes":9},"next":{"slug":"probability-calibration","title":"Making a 0.8 mean eighty per cent","minutes":9}}}