{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"catastrophic-forgetting","title":"Measuring what you broke","url":"https://addaly.com/learn/open-models/catastrophic-forgetting","position":58,"of":71,"minutes":8,"answer":"Fine-tuning moves the weights that held everything else, so report the task gain alongside a fifty-item regression set covering general instructions, other languages.","keyIdea":"Fine-tuning moves the weights that held everything else, so report the task gain alongside a fifty-item regression set covering general instructions, other languages, alternative formats and refusals. And mixing general data into training usually keeps most of the gain with a fraction of the loss."},"body":{"format":"markdown","text":"## What happens to everything else\n\nA fine-tune moves weights towards your examples. Those same weights encoded everything the model could already do. Move them far enough and previous capabilities degrade: this is catastrophic forgetting, and it is the reason a tuned model that scores brilliantly on your task can be quietly worse at half a dozen things you never measured.\n\nThe characteristic pattern after a small-dataset tune:\n\n- The task improves, sometimes a lot.\n- General instruction following weakens; the model answers in your trained format even when asked for something else.\n- Other languages degrade, often noticeably, because they were never in your data.\n- Refusal behaviour shifts in whichever direction your examples pushed it, including towards refusing nothing.\n- Formatting outside your trained format gets worse.\n- Long-context behaviour degrades if you trained on short examples only.\n\nLoRA reduces all of this compared with full fine-tuning, because far fewer parameters move. It does not eliminate it, and higher ranks forget more.\n\n## The regression set\n\nThe defence is one artefact, built once, and it takes about an hour.\n\nFifty items, covering what the model must not lose:\n\n- **10 general instruction-following items.** Summarise this, rewrite in a different tone, answer a general question, follow a two-part instruction.\n- **10 items in each other language you serve.** Even if the tune is English-only, especially if the tune is English-only.\n- **10 items with a different output format** from your trained one. If you trained JSON, ask for prose.\n- **10 safety and refusal items.** Things that should be refused, and things near the line that should not be.\n- **10 items from the original task at longer lengths** than your training examples.\n\nRun it against the base model first and record the answers. That is your reference. Run it after every tune and compare. Score by reading (fifty items takes twenty minutes) or with a rubric and a judge model, spot-checked.\n\nReport two numbers with every fine-tune: the task gain, and the regression change. A tune that gains 9 points on the task and loses 15 on the regression set is not an improvement, and without the second number it looks like one.\n\n## Reducing the damage\n\nIn order of effectiveness:\n\n**Fewer epochs.** Most forgetting arrives in later epochs, after the task has already been learned. Two epochs instead of five is usually free.\n\n**Lower rank.** Rank 8 forgets less than rank 64 and is enough for format and style tasks.\n\n**Mix in general data.** Add 10 to 30% general instruction-following examples to your training set, drawn from an open instruction dataset. This is the standard mitigation and it works well. The mixture keeps the model anchored while it learns your task.\n\n**Lower learning rate, or fewer steps.** Both reduce total movement.\n\n**Adapter rather than merge.** An unmerged adapter can be turned off. A model serving both the tuned and untuned behaviour, selecting per request, sidesteps the trade entirely. The shipping lesson covers how.\n\n## The measurement that decides\n\nRun three configurations on both sets: base model with your best prompt, tuned model, and tuned model with the mixed general data. Nine numbers, one afternoon.\n\n> Figure: Two configurations, measured on both sets. A two by two grid, rows Tuned on task data only and Tuned with 20% general data mixed in, columns Task score and Regression set: Tuned on task data only and Task score: +9 points; Tuned on task data only and Regression set: −15 points; Tuned with 20% general data mixed in and Task score: +8 points; Tuned with 20% general data mixed in and Regression set: −2 points. A tune that gains nine points on the task and loses fifteen on the regression set is not an improvement, and without the second number it looks like one. Run the base model with your best prompt as the third configuration, and ship the mixed run.\n\nThe result is usually that the mixed run gives almost all of the task gain with a small fraction of the regression, and that is the configuration to ship. Teams that skip this comparison ship the pure-task tune because it has the best headline number, and then spend a month diagnosing complaints that have nothing to do with the task they measured.\n\n> The honest framing: fine-tuning is not free capability, it is a trade. The regression set is how you find out what you traded, and it is the difference between a considered decision and an accident."},"check":{"question":"A fine-tune improves ticket-classification accuracy from 71% to 84% and the team ships it. Weeks later, support reports the assistant no longer handles Hindi messages well, though nothing about Hindi changed. What happened?","options":[{"text":"The tokenizer changed during fine-tuning, so Hindi is now segmented differently.","correct":false,"tempting":"Fine-tuning does not alter the tokenizer unless vocabulary was explicitly extended, which is a separate procedure."},{"text":"The English-only training data moved weights that also supported Hindi, degrading it, and no regression set was in place to catch it before release.","correct":true},{"text":"The adapter is being applied to Hindi requests when it should be disabled for them.","correct":false,"tempting":"Selective application is a real fix, but the degradation exists because the tune damaged the capability, not because the adapter is misrouted."},{"text":"Quantisation applied at deployment damaged Hindi more than English.","correct":false,"tempting":"A genuine effect from an earlier lesson, but it would have been present before the fine-tune too."}]},"prev":{"slug":"distilling-a-bigger-model","title":"Distilling a larger model into a smaller one","url":"https://addaly.com/learn/open-models/distilling-a-bigger-model"},"next":{"slug":"merging-models","title":"Merging models, and what it really does","url":"https://addaly.com/learn/open-models/merging-models"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Measuring what you broke', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/catastrophic-forgetting (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Measuring what you broke","courseTitle":"The Open Model Ecosystem","module":{"index":6,"slug":"adapting-a-model","title":"Adapting a model to your work"},"answer":null,"keyIdea":"Fine-tuning moves the weights that held everything else, so report the task gain alongside a fifty-item regression set covering general instructions, other languages, alternative formats and refusals. And mixing general data into training usually keeps most of the gain with a fraction of the loss.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"cols":["Task score","Regression set"],"rows":["Tuned on task data only","Tuned with 20% general data mixed in"],"type":"matrix","cells":[["+9 points","−15 points"],["+8 points","−2 points"]],"title":"Two configurations, measured on both sets","caption":"A tune that gains nine points on the task and loses fifteen on the regression set is not an improvement, and without the second number it looks like one. Run the base model with your best prompt as the third configuration, and ship the mixed run.","cellLabels":[["the best headline number","other languages and formats degrade"],["almost all of the gain","a fraction of the loss"]]},"alt":"Figure: Two configurations, measured on both sets. A two by two grid, rows Tuned on task data only and Tuned with 20% general data mixed in, columns Task score and Regression set: Tuned on task data only and Task score: +9 points; Tuned on task data only and Regression set: −15 points; Tuned with 20% general data mixed in and Task score: +8 points; Tuned with 20% general data mixed in and Regression set: −2 points. A tune that gains nine points on the task and loses fifteen on the regression set is not an improvement, and without the second number it looks like one. Run the base model with your best prompt as the third configuration, and ship the mixed run."}},"blocks":[{"t":"h2","id":"what-happens-to-everything-else","runs":[{"text":"What happens to everything else"}]},{"t":"p","runs":[{"text":"A fine-tune moves weights towards your examples. Those same weights encoded everything the model could already do. Move them far enough and previous capabilities degrade: this is catastrophic forgetting, and it is the reason a tuned model that scores brilliantly on your task can be quietly worse at half a dozen things you never measured."}]},{"t":"p","runs":[{"text":"The characteristic pattern after a small-dataset tune:"}]},{"t":"ul","items":[[{"text":"The task improves, sometimes a lot."}],[{"text":"General instruction following weakens; the model answers in your trained format even when asked for something else."}],[{"text":"Other languages degrade, often noticeably, because they were never in your data."}],[{"text":"Refusal behaviour shifts in whichever direction your examples pushed it, including towards refusing nothing."}],[{"text":"Formatting outside your trained format gets worse."}],[{"text":"Long-context behaviour degrades if you trained on short examples only."}]]},{"t":"p","runs":[{"text":"LoRA reduces all of this compared with full fine-tuning, because far fewer parameters move. It does not eliminate it, and higher ranks forget more."}]},{"t":"h2","id":"the-regression-set","runs":[{"text":"The regression set"}]},{"t":"p","runs":[{"text":"The defence is one artefact, built once, and it takes about an hour."}]},{"t":"p","runs":[{"text":"Fifty items, covering what the model must not lose:"}]},{"t":"ul","items":[[{"text":"10 general instruction-following items.","b":true},{"text":" Summarise this, rewrite in a different tone, answer a general question, follow a two-part instruction."}],[{"text":"10 items in each other language you serve.","b":true},{"text":" Even if the tune is English-only, especially if the tune is English-only."}],[{"text":"10 items with a different output format","b":true},{"text":" from your trained one. If you trained JSON, ask for prose."}],[{"text":"10 safety and refusal items.","b":true},{"text":" Things that should be refused, and things near the line that should not be."}],[{"text":"10 items from the original task at longer lengths","b":true},{"text":" than your training examples."}]]},{"t":"p","runs":[{"text":"Run it against the base model first and record the answers. That is your reference. Run it after every tune and compare. Score by reading (fifty items takes twenty minutes) or with a rubric and a judge model, spot-checked."}]},{"t":"p","runs":[{"text":"Report two numbers with every fine-tune: the task gain, and the regression change. A tune that gains 9 points on the task and loses 15 on the regression set is not an improvement, and without the second number it looks like one."}]},{"t":"h2","id":"reducing-the-damage","runs":[{"text":"Reducing the damage"}]},{"t":"p","runs":[{"text":"In order of effectiveness:"}]},{"t":"p","runs":[{"text":"Fewer epochs.","b":true},{"text":" Most forgetting arrives in later epochs, after the task has already been learned. Two epochs instead of five is usually free."}]},{"t":"p","runs":[{"text":"Lower rank.","b":true},{"text":" Rank 8 forgets less than rank 64 and is enough for format and style tasks."}]},{"t":"p","runs":[{"text":"Mix in general data.","b":true},{"text":" Add 10 to 30% general instruction-following examples to your training set, drawn from an open instruction dataset. This is the standard mitigation and it works well. The mixture keeps the model anchored while it learns your task."}]},{"t":"p","runs":[{"text":"Lower learning rate, or fewer steps.","b":true},{"text":" Both reduce total movement."}]},{"t":"p","runs":[{"text":"Adapter rather than merge.","b":true},{"text":" An unmerged adapter can be turned off. A model serving both the tuned and untuned behaviour, selecting per request, sidesteps the trade entirely. The shipping lesson covers how."}]},{"t":"h2","id":"the-measurement-that-decides","runs":[{"text":"The measurement that decides"}]},{"t":"p","runs":[{"text":"Run three configurations on both sets: base model with your best prompt, tuned model, and tuned model with the mixed general data. Nine numbers, one afternoon."}]},{"t":"p","runs":[{"text":"The result is usually that the mixed run gives almost all of the task gain with a small fraction of the regression, and that is the configuration to ship. Teams that skip this comparison ship the pure-task tune because it has the best headline number, and then spend a month diagnosing complaints that have nothing to do with the task they measured."}]},{"t":"quote","runs":[{"text":"The honest framing: fine-tuning is not free capability, it is a trade. The regression set is how you find out what you traded, and it is the difference between a considered decision and an accident."}]}],"check":{"question":[{"text":"A fine-tune improves ticket-classification accuracy from 71% to 84% and the team ships it. Weeks later, support reports the assistant no longer handles Hindi messages well, though nothing about Hindi changed. What happened?"}],"options":[{"text":[{"text":"The tokenizer changed during fine-tuning, so Hindi is now segmented differently."}],"correct":false,"tempting":[{"text":"Fine-tuning does not alter the tokenizer unless vocabulary was explicitly extended, which is a separate procedure."}]},{"text":[{"text":"The English-only training data moved weights that also supported Hindi, degrading it, and no regression set was in place to catch it before release."}],"correct":true,"tempting":null},{"text":[{"text":"The adapter is being applied to Hindi requests when it should be disabled for them."}],"correct":false,"tempting":[{"text":"Selective application is a real fix, but the degradation exists because the tune damaged the capability, not because the adapter is misrouted."}]},{"text":[{"text":"Quantisation applied at deployment damaged Hindi more than English."}],"correct":false,"tempting":[{"text":"A genuine effect from an earlier lesson, but it would have been present before the fine-tune too."}]}],"order":[0,3,2,1],"why":null},"prev":{"slug":"distilling-a-bigger-model","title":"Distilling a larger model into a smaller one","minutes":9},"next":{"slug":"merging-models","title":"Merging models, and what it really does","minutes":8}}}