{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"the-base-model-moved","title":"The day the base model moves","url":"https://addaly.com/learn/fine-tuning/the-base-model-moved","position":81,"of":84,"minutes":9,"answer":"An adapter fitted against one set of base weights loads cleanly onto a newer release and is silently worse, so pin the revision hash, and when a better base ships.","keyIdea":"An adapter fitted against one set of base weights loads cleanly onto a newer release and is silently worse, so pin the revision hash, and when a better base ships, first check whether it plus a good prompt already beats your tuned old model, because deleting the fine-tune is a success rather than a loss."},"body":{"format":"markdown","text":"## Adapters do not transfer\n\nA LoRA is a set of deltas fitted against one exact set of base weights. Apply it to different weights, even the next release in the same family, with the identical architecture, and you are perturbing matrices it was never fitted against.\n\nThe unpleasant part: **it loads without error**. Shapes match, the adapter attaches, the model generates. It is worse, in ways that look like a regression rather than a category error. There is no exception to catch.\n\nThe same risk exists within a version. Model repositories are edited in place: a tokenizer fix, a re-uploaded shard, a chat template correction. Pin the revision hash of the base model in your config, and record it beside the adapter, so that \"the same base model\" means the same bytes.\n\n## What does transfer\n\nEverything that matters, if you kept it:\n\n- The dataset, with its filters and its data card.\n- The evaluation set, the canary set and the refusal set.\n- The configuration, including the learning rate and target modules you arrived at.\n- The ablation log, which tells you what does and does not help on this data.\n\nThe model was always the perishable output. These are the durable ones, and a team that has them treats a base-model release as an afternoon.\n\n## The decision at each release\n\nWhen a meaningfully better base ships, which for open-weight families has historically been every six to nine months, work through four questions in order.\n\n**1. Does the new base, with a good prompt, already beat our tuned old model?**\n\nRun it. This takes an hour and it is often yes, particularly when your fine-tune was addressing instruction-following or format adherence, which is exactly what each generation of base models does better.\n\nIf yes, the correct action is to delete the fine-tune and ship the prompt. That is a success (it means the capability you paid to install is now free) and it is regularly resisted for reasons that have more to do with sunk effort than with the product.\n\n**2. Does the new base tokenise our language better?** For non-English work this is often the largest single improvement in a release, and it changes cost and context as well as quality.\n\n**3. What does retraining cost us?** If the pipeline is one command, this is not a real question. If nobody has run the rebuild in eight months, find out now rather than under pressure.\n\n**4. Is anything in the interface different?** A new chat template, new special tokens, a different system-prompt convention. These do not break loudly; they degrade.\n\n## The rebuild target\n\nThe answer to question three should be a command:\n\n```makefile\nrebuild: dataset train merge quantise evaluate\n\t@echo \"artefact: out/merged-q4km.gguf\"\n```\n\nRun it once a quarter against the *current* base whether or not you need to. A pipeline that is exercised regularly works; one that is exercised annually does not, and you discover that at the worst moment.\n\nTime it and write the number in the decision record. \"Rebuild: 4 hours, unattended\" changes how an organisation reasons about its dependency on a model.\n\n## The migration itself\n\n1. Retrain on the new base with the unchanged dataset and config.\n2. Evaluate old and new on the same held-out set, canary set and refusal set.\n3. Shadow the new model against live traffic without serving its output, comparing distributions of length, refusal rate and fallback rate.\n4. Roll out to a slice, watch the user-behaviour proxies, then complete.\n5. Keep the old artefact for a rollback window.\n\nAnd re-run the cost table, because the new base may be a different size and change the arithmetic that justified self-hosting.\n\n> Figure: Four questions, in order, when a better base ships. Timeline: First, Does the new base, with a good prompt, already beat our tuned old model? An hour to find out, and the answer is often yes.; Second, Does it tokenise our language better? For non-English work this is often the largest single improvement in a release.; Third, What does retraining cost us? If nobody has run the rebuild in eight months, find out now rather than under pressure.; Fourth, Is anything in the interface different: a new chat template, new special tokens, a changed system-prompt convention? These degrade rather than break.. Deleting the fine-tune because the base caught up is a success: the capability you paid to install is now free. It is regularly resisted for reasons that have more to do with sunk effort than with the product.\n\n## The framing worth keeping\n\nA fine-tune is not a thing you make. It is a thing you maintain, on a cadence set by somebody else's release schedule.\n\nThat is the honest closing argument of module one, arriving here as a practical matter. The projects that survive base-model churn are the ones whose durable artefacts (dataset, evaluation, configuration, decision record) were treated as the deliverable, with the weights as a build output. The projects that do not survive it are the ones where somebody produced a model."},"check":{"question":"A team loads their existing adapter onto a newer release of the same model family. It loads without error, generates fluently, and performs worse than the old combination. Why is no error raised?","options":[{"text":"PEFT validates only tensor shapes, which match across the release, so nothing detects that the underlying weights differ.","correct":true},{"text":"The adapter is applied at inference rather than merged, and unmerged adapters skip the compatibility check that merging performs.","correct":false,"tempting":"Invents a check that merging does not perform either; merging adds deltas to whatever matrix it is given."},{"text":"Adapters store a hash of the base model, and a mismatch produces a warning rather than an error by default.","correct":false,"tempting":"Assumes an integrity field that adapter configs record as a name rather than verify as a hash."},{"text":"The new release is backwards compatible by design, so the degradation comes from the changed chat template rather than the weights.","correct":false,"tempting":"Names a real hazard in releases, but a template change would be fixable while the weight mismatch would not be."}]},"prev":{"slug":"monitoring-a-tuned-model","title":"Monitoring a model that fails silently","url":"https://addaly.com/learn/fine-tuning/monitoring-a-tuned-model"},"next":{"slug":"versioning-models-and-data","title":"Versioning so a model can be explained later","url":"https://addaly.com/learn/fine-tuning/versioning-models-and-data"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The day the base model moves', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/the-base-model-moved (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The day the base model moves","courseTitle":"Fine-Tuning, and When Not To","module":{"index":9,"slug":"shipping-and-living-with-it","title":"Shipping it, and living with it"},"answer":null,"keyIdea":"An adapter fitted against one set of base weights loads cleanly onto a newer release and is silently worse, so pin the revision hash, and when a better base ships, first check whether it plus a good prompt already beats your tuned old model, because deleting the fine-tune is a success rather than a loss.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"timeline","title":"Four questions, in order, when a better base ships","events":[{"when":"First","what":"Does the new base, with a good prompt, already beat our tuned old model? An hour to find out, and the answer is often yes."},{"when":"Second","what":"Does it tokenise our language better? For non-English work this is often the largest single improvement in a release."},{"when":"Third","what":"What does retraining cost us? If nobody has run the rebuild in eight months, find out now rather than under pressure."},{"when":"Fourth","what":"Is anything in the interface different: a new chat template, new special tokens, a changed system-prompt convention? These degrade rather than break."}],"caption":"Deleting the fine-tune because the base caught up is a success: the capability you paid to install is now free. It is regularly resisted for reasons that have more to do with sunk effort than with the product."},"alt":"Figure: Four questions, in order, when a better base ships. Timeline: First, Does the new base, with a good prompt, already beat our tuned old model? An hour to find out, and the answer is often yes.; Second, Does it tokenise our language better? For non-English work this is often the largest single improvement in a release.; Third, What does retraining cost us? If nobody has run the rebuild in eight months, find out now rather than under pressure.; Fourth, Is anything in the interface different: a new chat template, new special tokens, a changed system-prompt convention? These degrade rather than break.. Deleting the fine-tune because the base caught up is a success: the capability you paid to install is now free. It is regularly resisted for reasons that have more to do with sunk effort than with the product."}},"blocks":[{"t":"h2","id":"adapters-do-not-transfer","runs":[{"text":"Adapters do not transfer"}]},{"t":"p","runs":[{"text":"A LoRA is a set of deltas fitted against one exact set of base weights. Apply it to different weights, even the next release in the same family, with the identical architecture, and you are perturbing matrices it was never fitted against."}]},{"t":"p","runs":[{"text":"The unpleasant part: "},{"text":"it loads without error","b":true},{"text":". Shapes match, the adapter attaches, the model generates. It is worse, in ways that look like a regression rather than a category error. There is no exception to catch."}]},{"t":"p","runs":[{"text":"The same risk exists within a version. Model repositories are edited in place: a tokenizer fix, a re-uploaded shard, a chat template correction. Pin the revision hash of the base model in your config, and record it beside the adapter, so that “the same base model” means the same bytes."}]},{"t":"h2","id":"what-does-transfer","runs":[{"text":"What does transfer"}]},{"t":"p","runs":[{"text":"Everything that matters, if you kept it:"}]},{"t":"ul","items":[[{"text":"The dataset, with its filters and its data card."}],[{"text":"The evaluation set, the canary set and the refusal set."}],[{"text":"The configuration, including the learning rate and target modules you arrived at."}],[{"text":"The ablation log, which tells you what does and does not help on this data."}]]},{"t":"p","runs":[{"text":"The model was always the perishable output. These are the durable ones, and a team that has them treats a base-model release as an afternoon."}]},{"t":"h2","id":"the-decision-at-each-release","runs":[{"text":"The decision at each release"}]},{"t":"p","runs":[{"text":"When a meaningfully better base ships, which for open-weight families has historically been every six to nine months, work through four questions in order."}]},{"t":"p","runs":[{"text":"1. Does the new base, with a good prompt, already beat our tuned old model?","b":true}]},{"t":"p","runs":[{"text":"Run it. This takes an hour and it is often yes, particularly when your fine-tune was addressing instruction-following or format adherence, which is exactly what each generation of base models does better."}]},{"t":"p","runs":[{"text":"If yes, the correct action is to delete the fine-tune and ship the prompt. That is a success (it means the capability you paid to install is now free) and it is regularly resisted for reasons that have more to do with sunk effort than with the product."}]},{"t":"p","runs":[{"text":"2. Does the new base tokenise our language better?","b":true},{"text":" For non-English work this is often the largest single improvement in a release, and it changes cost and context as well as quality."}]},{"t":"p","runs":[{"text":"3. What does retraining cost us?","b":true},{"text":" If the pipeline is one command, this is not a real question. If nobody has run the rebuild in eight months, find out now rather than under pressure."}]},{"t":"p","runs":[{"text":"4. Is anything in the interface different?","b":true},{"text":" A new chat template, new special tokens, a different system-prompt convention. These do not break loudly; they degrade."}]},{"t":"h2","id":"the-rebuild-target","runs":[{"text":"The rebuild target"}]},{"t":"p","runs":[{"text":"The answer to question three should be a command:"}]},{"t":"code","lang":"makefile","label":"Makefile","text":"rebuild: dataset train merge quantise evaluate\n\t@echo \"artefact: out/merged-q4km.gguf\""},{"t":"p","runs":[{"text":"Run it once a quarter against the "},{"text":"current","i":true},{"text":" base whether or not you need to. A pipeline that is exercised regularly works; one that is exercised annually does not, and you discover that at the worst moment."}]},{"t":"p","runs":[{"text":"Time it and write the number in the decision record. “Rebuild: 4 hours, unattended” changes how an organisation reasons about its dependency on a model."}]},{"t":"h2","id":"the-migration-itself","runs":[{"text":"The migration itself"}]},{"t":"ol","start":1,"items":[[{"text":"Retrain on the new base with the unchanged dataset and config."}],[{"text":"Evaluate old and new on the same held-out set, canary set and refusal set."}],[{"text":"Shadow the new model against live traffic without serving its output, comparing distributions of length, refusal rate and fallback rate."}],[{"text":"Roll out to a slice, watch the user-behaviour proxies, then complete."}],[{"text":"Keep the old artefact for a rollback window."}]]},{"t":"p","runs":[{"text":"And re-run the cost table, because the new base may be a different size and change the arithmetic that justified self-hosting."}]},{"t":"h2","id":"the-framing-worth-keeping","runs":[{"text":"The framing worth keeping"}]},{"t":"p","runs":[{"text":"A fine-tune is not a thing you make. It is a thing you maintain, on a cadence set by somebody else’s release schedule."}]},{"t":"p","runs":[{"text":"That is the honest closing argument of module one, arriving here as a practical matter. The projects that survive base-model churn are the ones whose durable artefacts (dataset, evaluation, configuration, decision record) were treated as the deliverable, with the weights as a build output. The projects that do not survive it are the ones where somebody produced a model."}]}],"check":{"question":[{"text":"A team loads their existing adapter onto a newer release of the same model family. It loads without error, generates fluently, and performs worse than the old combination. Why is no error raised?"}],"options":[{"text":[{"text":"PEFT validates only tensor shapes, which match across the release, so nothing detects that the underlying weights differ."}],"correct":true,"tempting":null},{"text":[{"text":"The adapter is applied at inference rather than merged, and unmerged adapters skip the compatibility check that merging performs."}],"correct":false,"tempting":[{"text":"Invents a check that merging does not perform either; merging adds deltas to whatever matrix it is given."}]},{"text":[{"text":"Adapters store a hash of the base model, and a mismatch produces a warning rather than an error by default."}],"correct":false,"tempting":[{"text":"Assumes an integrity field that adapter configs record as a name rather than verify as a hash."}]},{"text":[{"text":"The new release is backwards compatible by design, so the degradation comes from the changed chat template rather than the weights."}],"correct":false,"tempting":[{"text":"Names a real hazard in releases, but a template change would be fixable while the weight mismatch would not be."}]}],"order":[2,3,0,1],"why":null},"prev":{"slug":"monitoring-a-tuned-model","title":"Monitoring a model that fails silently","minutes":9},"next":{"slug":"versioning-models-and-data","title":"Versioning so a model can be explained later","minutes":8}}}