{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"calibration-and-overconfidence","title":"Calibration after tuning","url":"https://addaly.com/learn/fine-tuning/calibration-and-overconfidence","position":74,"of":84,"minutes":8,"answer":"Fine-tuning sharpens the output distribution and preference tuning rewards confidence, so a tuned model's 0.9 no longer means 0.9, which breaks every threshold.","keyIdea":"Fine-tuning sharpens the output distribution and preference tuning rewards confidence, so a tuned model's 0.9 no longer means 0.9, which breaks every threshold, fallback and abstention rule downstream until temperature scaling is refitted on the exact artefact you serve."},"body":{"format":"markdown","text":"## Confidence stops meaning what it meant\n\nA well-calibrated model's confidence tracks its accuracy: among the things it says with 80% confidence, about 80% are right.\n\nFine-tuning degrades this, reliably and in one direction. Training sharpens the output distribution (that is what maximising the likelihood of a target does) so the model becomes more confident about everything, including the things it is wrong about. Preference tuning compounds it, because judges and annotators prefer confident answers to hedged ones, so confidence is directly rewarded.\n\nThe result: a model whose 0.9 used to mean 0.9 and now means 0.75.\n\n## Why you should care\n\nIf nothing downstream reads the confidence, you can ignore this. The moment anything does, it matters completely:\n\n- \"Fall back to the expensive model below 0.7\": a rule about a number that no longer means what it did, so your fallback fires on the wrong items and your cost model is wrong.\n- \"Auto-approve above 0.95\": now auto-approving cases the model is not actually sure about.\n- \"Say 'I am not certain' when confidence is low\": the model stops saying it, because its confidence stopped being low.\n\nEvery abstention and routing design in this course depends on calibration, and fine-tuning is the step that breaks it.\n\n## Measuring it\n\nFor a classifier the measurement is direct. Bin held-out predictions by confidence and compare each bin's mean confidence with its accuracy:\n\n```python\nbins = np.linspace(0, 1, 11)\nidx  = np.digitize(conf, bins) - 1\nece  = 0.0\nfor b in range(10):\n    m = idx == b\n    if m.sum():\n        ece += m.mean() * abs(acc[m].mean() - conf[m].mean())\nprint(f\"ECE = {ece:.3f}\")\n```\n\nExpected calibration error is the average gap, weighted by how many items fall in each bin. Under about 0.05 is good; above 0.15 means the numbers cannot carry a threshold. Plotting the same thing (accuracy against confidence, with the diagonal drawn) shows you the *shape* of the miscalibration, which the single number hides.\n\nFor a generative model there are two usable proxies:\n\n- **Sequence probability** of the answer it produced, though this confounds confidence with length and with how many ways there were to phrase the same answer.\n- **Verbalised confidence**: ask the model for a number, then check whether that number tracks accuracy. Crude, surprisingly serviceable, and it is what most abstention designs end up using.\n\n## Fixing it\n\n**Temperature scaling.** Fit a single scalar on a validation set, divide the logits by it before the softmax. One parameter, ten lines of code, and it corrects most of the gap without changing any prediction: the ranking is preserved, only the confidences move. Use it on any classifier whose confidence feeds a decision.\n\n**Train on uncertainty.** Include examples where the correct answer is \"I do not have that information\", \"this could be either A or B\", or \"I would need the order number to answer\". If your dataset contains no uncertainty, the model has never been shown that uncertainty is an available output. This is the same mechanism as the refusal problem in the previous lesson, and the same fix.\n\n**Recalibrate after every change.** Quantisation shifts the logits. Merging an adapter shifts them. A new base model shifts them. The scalar you fitted in March is fitted to a model that no longer exists.\n\n## Abstention is a separate decision\n\nOne clarification that saves confusion. Calibration tells you whether a confidence is honest. It does not tell you where to put the threshold: that is a cost question, and it belongs to whoever owns the consequences.\n\nIf a wrong routing decision costs £4 to correct and an escalation to the expensive path costs £0.30, the threshold sits wherever expected cost is minimised, which on a calibrated model you can compute directly rather than guess:\n\n```python\nbest = min(thresholds, key=lambda t:\n           0.30 * (conf < t).mean() + 4.00 * ((conf >= t) & wrong).mean())\n```\n\nThat is two numbers a business can supply and one line of code. The common failure is to pick 0.7 because it sounds cautious, on a model whose 0.7 has never been checked, and to call the result a policy.\n\n## The rule to carry\n\nIf a number in your system crosses a threshold to make a decision, that number needs a calibration check on the exact artefact you serve, refreshed whenever the artefact changes.\n\nAnd if you have not checked it, do not build the threshold. A confidence-based fallback on an uncalibrated model is not a safety mechanism; it is a mechanism that fires at a rate you have not measured, which is worse than no mechanism because it is believed."},"check":{"question":"A routing system sends anything below 0.7 confidence to an expensive model. After fine-tuning, task accuracy improved but the expensive fallback almost never fires and overall error rose. What happened?","options":[{"text":"The fine-tune improved accuracy on easy cases only, so the hard cases that used to be routed are now handled badly by the small model.","correct":false,"tempting":"Describes a plausible accuracy shift but does not explain why the routing rule stopped triggering on those same hard cases."},{"text":"The threshold should be recomputed for every new model, since confidence scales with the number of classes and the class count changed.","correct":false,"tempting":"Attributes the shift to a class-count dependency that does not apply when the label set is unchanged."},{"text":"Training sharpened the output distribution, so confidences rose across the board and uncertain cases now score above the threshold.","correct":true},{"text":"Temperature scaling was applied during training, which compressed the confidence range and pushed all values upward.","correct":false,"tempting":"Inverts what temperature scaling does and when it is applied; it is fitted after training and would reduce rather than inflate confidence."}]},"prev":{"slug":"safety-regression-after-tuning","title":"Safety regression, including from benign data","url":"https://addaly.com/learn/fine-tuning/safety-regression-after-tuning"},"next":{"slug":"ablations-that-earn-their-cost","title":"Ablations that earn their cost","url":"https://addaly.com/learn/fine-tuning/ablations-that-earn-their-cost"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Calibration after tuning', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/calibration-and-overconfidence (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Calibration after tuning","courseTitle":"Fine-Tuning, and When Not To","module":{"index":8,"slug":"did-it-actually-work","title":"Did it actually work"},"answer":null,"keyIdea":"Fine-tuning sharpens the output distribution and preference tuning rewards confidence, so a tuned model’s 0.9 no longer means 0.9, which breaks every threshold, fallback and abstention rule downstream until temperature scaling is refitted on the exact artefact you serve.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"confidence-stops-meaning-what-it-meant","runs":[{"text":"Confidence stops meaning what it meant"}]},{"t":"p","runs":[{"text":"A well-calibrated model’s confidence tracks its accuracy: among the things it says with 80% confidence, about 80% are right."}]},{"t":"p","runs":[{"text":"Fine-tuning degrades this, reliably and in one direction. Training sharpens the output distribution (that is what maximising the likelihood of a target does) so the model becomes more confident about everything, including the things it is wrong about. Preference tuning compounds it, because judges and annotators prefer confident answers to hedged ones, so confidence is directly rewarded."}]},{"t":"p","runs":[{"text":"The result: a model whose 0.9 used to mean 0.9 and now means 0.75."}]},{"t":"h2","id":"why-you-should-care","runs":[{"text":"Why you should care"}]},{"t":"p","runs":[{"text":"If nothing downstream reads the confidence, you can ignore this. The moment anything does, it matters completely:"}]},{"t":"ul","items":[[{"text":"“Fall back to the expensive model below 0.7”: a rule about a number that no longer means what it did, so your fallback fires on the wrong items and your cost model is wrong."}],[{"text":"“Auto-approve above 0.95”: now auto-approving cases the model is not actually sure about."}],[{"text":"“Say ‘I am not certain’ when confidence is low”: the model stops saying it, because its confidence stopped being low."}]]},{"t":"p","runs":[{"text":"Every abstention and routing design in this course depends on calibration, and fine-tuning is the step that breaks it."}]},{"t":"h2","id":"measuring-it","runs":[{"text":"Measuring it"}]},{"t":"p","runs":[{"text":"For a classifier the measurement is direct. Bin held-out predictions by confidence and compare each bin’s mean confidence with its accuracy:"}]},{"t":"code","lang":"python","label":"Python","text":"bins = np.linspace(0, 1, 11)\nidx  = np.digitize(conf, bins) - 1\nece  = 0.0\nfor b in range(10):\n    m = idx == b\n    if m.sum():\n        ece += m.mean() * abs(acc[m].mean() - conf[m].mean())\nprint(f\"ECE = {ece:.3f}\")"},{"t":"p","runs":[{"text":"Expected calibration error is the average gap, weighted by how many items fall in each bin. Under about 0.05 is good; above 0.15 means the numbers cannot carry a threshold. Plotting the same thing (accuracy against confidence, with the diagonal drawn) shows you the "},{"text":"shape","i":true},{"text":" of the miscalibration, which the single number hides."}]},{"t":"p","runs":[{"text":"For a generative model there are two usable proxies:"}]},{"t":"ul","items":[[{"text":"Sequence probability","b":true},{"text":" of the answer it produced, though this confounds confidence with length and with how many ways there were to phrase the same answer."}],[{"text":"Verbalised confidence","b":true},{"text":": ask the model for a number, then check whether that number tracks accuracy. Crude, surprisingly serviceable, and it is what most abstention designs end up using."}]]},{"t":"h2","id":"fixing-it","runs":[{"text":"Fixing it"}]},{"t":"p","runs":[{"text":"Temperature scaling.","b":true},{"text":" Fit a single scalar on a validation set, divide the logits by it before the softmax. One parameter, ten lines of code, and it corrects most of the gap without changing any prediction: the ranking is preserved, only the confidences move. Use it on any classifier whose confidence feeds a decision."}]},{"t":"p","runs":[{"text":"Train on uncertainty.","b":true},{"text":" Include examples where the correct answer is “I do not have that information”, “this could be either A or B”, or “I would need the order number to answer”. If your dataset contains no uncertainty, the model has never been shown that uncertainty is an available output. This is the same mechanism as the refusal problem in the previous lesson, and the same fix."}]},{"t":"p","runs":[{"text":"Recalibrate after every change.","b":true},{"text":" Quantisation shifts the logits. Merging an adapter shifts them. A new base model shifts them. The scalar you fitted in March is fitted to a model that no longer exists."}]},{"t":"h2","id":"abstention-is-a-separate-decision","runs":[{"text":"Abstention is a separate decision"}]},{"t":"p","runs":[{"text":"One clarification that saves confusion. Calibration tells you whether a confidence is honest. It does not tell you where to put the threshold: that is a cost question, and it belongs to whoever owns the consequences."}]},{"t":"p","runs":[{"text":"If a wrong routing decision costs £4 to correct and an escalation to the expensive path costs £0.30, the threshold sits wherever expected cost is minimised, which on a calibrated model you can compute directly rather than guess:"}]},{"t":"code","lang":"python","label":"Python","text":"best = min(thresholds, key=lambda t:\n           0.30 * (conf < t).mean() + 4.00 * ((conf >= t) & wrong).mean())"},{"t":"p","runs":[{"text":"That is two numbers a business can supply and one line of code. The common failure is to pick 0.7 because it sounds cautious, on a model whose 0.7 has never been checked, and to call the result a policy."}]},{"t":"h2","id":"the-rule-to-carry","runs":[{"text":"The rule to carry"}]},{"t":"p","runs":[{"text":"If a number in your system crosses a threshold to make a decision, that number needs a calibration check on the exact artefact you serve, refreshed whenever the artefact changes."}]},{"t":"p","runs":[{"text":"And if you have not checked it, do not build the threshold. A confidence-based fallback on an uncalibrated model is not a safety mechanism; it is a mechanism that fires at a rate you have not measured, which is worse than no mechanism because it is believed."}]}],"check":{"question":[{"text":"A routing system sends anything below 0.7 confidence to an expensive model. After fine-tuning, task accuracy improved but the expensive fallback almost never fires and overall error rose. What happened?"}],"options":[{"text":[{"text":"The fine-tune improved accuracy on easy cases only, so the hard cases that used to be routed are now handled badly by the small model."}],"correct":false,"tempting":[{"text":"Describes a plausible accuracy shift but does not explain why the routing rule stopped triggering on those same hard cases."}]},{"text":[{"text":"The threshold should be recomputed for every new model, since confidence scales with the number of classes and the class count changed."}],"correct":false,"tempting":[{"text":"Attributes the shift to a class-count dependency that does not apply when the label set is unchanged."}]},{"text":[{"text":"Training sharpened the output distribution, so confidences rose across the board and uncertain cases now score above the threshold."}],"correct":true,"tempting":null},{"text":[{"text":"Temperature scaling was applied during training, which compressed the confidence range and pushed all values upward."}],"correct":false,"tempting":[{"text":"Inverts what temperature scaling does and when it is applied; it is fitted after training and would reduce rather than inflate confidence."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"safety-regression-after-tuning","title":"Safety regression, including from benign data","minutes":9},"next":{"slug":"ablations-that-earn-their-cost","title":"Ablations that earn their cost","minutes":8}}}