Moving the threshold is a business decision

The model gives a score; you choose the line

Most classifiers do not output yes or no. They output a number between 0 and 1, and somebody (usually by default, usually without noticing) decided that 0.5 means yes.

0.5 is not a principled choice. It is the midpoint of a range, and the range is arbitrary. The threshold is where you convert a score into an action, and it should be set by what the two kinds of mistake cost you.

Take the content filter from the previous lesson: 1,000 posts, 60 harmful. Sweep the threshold and the picture changes completely.

• At 0.3: flags 210 posts, catches 57 of 60. Recall 95%, precision 27%. The review queue has 153 innocent posts in it.
• At 0.5: flags 80, catches 48. Recall 80%, precision 60%.
• At 0.8: flags 38, catches 34. Recall 57%, precision 89%. Twenty-six harmful posts go through.

Same model, same weights, three different products. Nobody retrained anything.

Put a number on each mistake

You can do better than arguing about it. Assign a cost to a false positive and a false negative, in any consistent unit, and pick the threshold that minimises total cost.

Say a false positive costs you five minutes of a moderator’s time, and a false negative costs the equivalent of two hours of harm, complaints and trust. That is a ratio of 24 to 1.

cost = FP * 5 + FN * 120     (in minutes)

threshold 0.3:  153*5 + 3*120  =  765 + 360  = 1125
threshold 0.5:   32*5 + 12*120 =  160 + 1440 = 1600
threshold 0.8:    4*5 + 26*120 =   20 + 3120 = 3140

The low threshold wins by a wide margin, and now the argument is about the 24:1 ratio, which is the argument you should have been having all along. People who will not agree on a threshold will often agree on a ratio, and the ratio is the honest form of the disagreement.

The numbers are made up. Making them up explicitly, in a document, is still much better than leaving them implicit in a default.

PR curves and the ROC trap

Sweeping the threshold and plotting the results gives you a curve, and there are two conventions.

The ROC curve plots true positive rate against false positive rate, and its area (AUC) is the number most commonly quoted. Under heavy imbalance it is misleading, because the false positive rate has 940 true negatives in its denominator. Going from 32 to 153 false positives moves that rate from 3.4% to 16%; the curve barely notices, while your review queue has grown fivefold.

The precision–recall curve puts precision on one axis, and precision has the flagged count in its denominator, so it feels every one of those false positives. For rare positives, read the PR curve. Report average precision rather than ROC-AUC.

Both are one line in scikit-learn (precision_recall_curve, roc_curve), free, and worth plotting once even if you never plot it again: seeing your own curve’s shape tells you whether there is a threshold that gets you both, or whether the model is not separating the classes and no threshold will save you.

Scores are not probabilities

A score of 0.9 does not mean “right 90% of the time” unless the model has been calibrated. Check it directly: bucket predictions into ten bins by score, and in each bin compare the mean predicted score to the observed fraction of positives. If the 0.9 bucket is right 62% of the time, your scores are inflated, which is normal for neural networks.

Fixing it is cheap: Platt scaling (a logistic regression on the scores) or isotonic regression, both in sklearn.calibration, both needing only a held-out set with labels. Calibrate before you compute expected cost, because the cost arithmetic above assumes the score means something.

For a large language model asked to rate its own confidence, none of this applies and calibration is worse than poor. Self-reported confidence clusters near 0.9 regardless of correctness. If you need a usable score from an LLM classifier, take the token log-probabilities of the label tokens where the provider exposes them, and calibrate those. Treat a number the model has typed out in prose as text, not as a probability.

Two thresholds are often better than one

The most useful shape in practice is not one line but two. Below 0.3, act automatically as negative. Above 0.85, act automatically as positive. Between, send it to a human.

This lets you tune the middle band to your actual review capacity: if the queue can absorb 200 items a day, widen or narrow the band until it does. You get high precision on both automatic paths and you spend your scarce human attention exactly where the model is unsure. It is also the design that degrades gracefully, because when volume spikes the band can be narrowed rather than the whole system switched off.