{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"the-cost-of-owning-a-model","title":"The cost of owning a model","url":"https://addaly.com/learn/fine-tuning/the-cost-of-owning-a-model","position":7,"of":84,"minutes":9,"answer":"The training run is the cheapest part of a fine-tune; the real cost is a dataset, an evaluation set, a serving line item and a rebuild every time the base model is superseded.","keyIdea":"The training run is the cheapest part of a fine-tune; the real cost is a dataset, an evaluation set, a serving line item and a rebuild every time the base model is superseded, so the thing you must be able to reproduce is the pipeline, not the weights."},"body":{"format":"markdown","text":"## The GPU bill is the small number\n\nAsk somebody what a fine-tune costs and they will quote the training run: four dollars, eleven dollars, a free Colab session. That number is real and it is almost irrelevant. What you are buying is not a run, it is an artefact with a maintenance schedule attached.\n\nHere is the fuller bill for one modest production fine-tune, with the parts nobody quotes.\n\n**One-off, before you ship**\n\n- Dataset construction: 2,000 examples, of which maybe 300 are hand-written or hand-corrected. At five minutes each that is 25 hours, and that is the optimistic version.\n- Evaluation set construction: 200 items, labelled, held out, plus a canary set for general capability. Another 15 hours, and it must come first.\n- The training runs themselves: not one, but eight to fifteen, because you will change the template, find a masking bug, try two learning rates and two ranks. Call it $40 of GPU rather than $4.\n- Evaluation of each candidate: cheap per run, expensive in attention.\n\n**Recurring, for as long as it lives**\n\n- Serving. Either a card that idles at 3 a.m. or a per-token bill from a provider that hosts adapters. A single always-on L4 is roughly $450 a month; a serverless endpoint is cheaper at low volume and has cold starts you will be asked about.\n- Monitoring. Someone has to look at the outputs, because a fine-tuned model degrades silently as the input distribution moves.\n- Retraining. Which brings us to the real cost.\n\n## The base model moves under you\n\nA LoRA adapter is a set of weight deltas for one specific set of base weights. It does not transfer. When the base model you trained against is superseded, and open-weight families have historically shipped a meaningful upgrade every six to nine months, you have three choices, all of them costs:\n\n> Figure: One modest fine-tune, before it ships. Bars (hours of a person's time): Dataset, 2,000 examples with 300 hand-corrected 25; Evaluation set and canary set 15; Reading candidate outputs and choosing 10; Config, masking bugs, two learning rates 8; The GPU runs themselves 1. The GPU bill for eight to fifteen runs is about forty dollars. Everything else on this chart is paid in somebody's attention, and the evaluation set has to come first.\n\n1. **Stay on the old base.** Your product's quality floor is now frozen at the level of a model from last year, while the free alternative everyone else prompts keeps improving.\n2. **Retrain onto the new base.** Cheap if your dataset and pipeline are intact, expensive if they are a notebook on a laptop belonging to somebody who has left.\n3. **Re-evaluate whether you still need the fine-tune at all.** Frequently the honest answer is no: the new base does with a prompt what the old base needed training for. This happens often enough that it should be a scheduled question, not a surprise.\n\nThe rule that falls out: keep the dataset, the config, and the evaluation set in version control as first-class artefacts, because the model is the *output* of those things and will be regenerated several times. A team that can rebuild its fine-tune in an afternoon treats a base model upgrade as an improvement. A team that cannot treats it as a threat.\n\n## The second, quieter cost\n\nA fine-tuned model is a dependency your organisation does not know how to reason about. Ask three practical questions about the one you are proposing:\n\n- If the person who trained it leaves next month, can somebody else reproduce it from what is written down?\n- When a customer complains about one specific output, what is the debugging procedure? For a prompt, you read the prompt. For weights, you look at the dataset and guess.\n- When legal asks what the model was trained on, is there an answer?\n\nIf the answers are no, nobody knows, and not really, the fine-tune is not an asset, it is a liability that currently performs well.\n\n## Where the sums work\n\nNone of this is an argument against fine-tuning. It is an argument for computing the right number. The cases where the full cost is clearly worth it share a shape:\n\n- **High volume.** The saving is per-request, so the case strengthens linearly with traffic and the fixed costs amortise. Below roughly a hundred thousand requests a month it is hard to make the arithmetic work on cost alone.\n- **A stable task.** Something whose definition will not change next quarter: transcript formatting, a fixed extraction schema, one language pair.\n- **A constraint no prompt satisfies.** Latency budgets under 200 ms, on-device deployment, an air-gapped environment, or data that may not leave the building.\n\nWrite the monthly saving and the annual maintenance on the same line before you start. If the saving is $900 a month and the maintenance is a day of engineering a month, you have not saved anything; you have converted a bill into a chore."},"check":{"question":"A team's LoRA adapter for Llama-3.1-8B performs well. Nine months later a markedly better base model of the same family ships. Why can they not simply load their adapter onto the new base?","options":[{"text":"The adapter encodes deltas against one exact set of base weights, which different weights no longer match.","correct":true},{"text":"Adapters are stored in a format tied to the library version, so an older PEFT checkpoint cannot be read by newer code.","correct":false,"tempting":"Confuses a format-compatibility annoyance, which is usually fixable, with the substantive reason the weights are meaningless on a different base."},{"text":"The licence of the new base forbids loading adapters trained against earlier releases.","correct":false,"tempting":"Invents a licence term; the obstacle is mechanical, not legal, and would exist between two permissively licensed models."},{"text":"Adapters only work with the tokenizer they were trained with, and a new model release always ships a new tokenizer.","correct":false,"tempting":"Picks a genuine hazard that is neither necessary nor sufficient here; tokenizers are often unchanged within a family and the adapter would still not transfer."}]},"prev":{"slug":"distillation-as-the-real-case","title":"Distillation, the case that usually survives","url":"https://addaly.com/learn/fine-tuning/distillation-as-the-real-case"},"next":{"slug":"licences-and-what-you-may-train-on","title":"Licences, and what you may train on","url":"https://addaly.com/learn/fine-tuning/licences-and-what-you-may-train-on"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The cost of owning a model', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/the-cost-of-owning-a-model (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The cost of owning a model","courseTitle":"Fine-Tuning, and When Not To","module":{"index":1,"slug":"deciding-whether-to-fine-tune","title":"Deciding whether to fine-tune at all"},"answer":null,"keyIdea":"The training run is the cheapest part of a fine-tune; the real cost is a dataset, an evaluation set, a serving line item and a rebuild every time the base model is superseded, so the thing you must be able to reproduce is the pipeline, not the weights.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"hours of a person’s time","title":"One modest fine-tune, before it ships","series":[{"label":"Dataset, 2,000 examples with 300 hand-corrected","value":25,"highlight":true},{"label":"Evaluation set and canary set","value":15},{"label":"Reading candidate outputs and choosing","value":10},{"label":"Config, masking bugs, two learning rates","value":8},{"label":"The GPU runs themselves","value":1}],"caption":"The GPU bill for eight to fifteen runs is about forty dollars. Everything else on this chart is paid in somebody’s attention, and the evaluation set has to come first."},"alt":"Figure: One modest fine-tune, before it ships. Bars (hours of a person’s time): Dataset, 2,000 examples with 300 hand-corrected 25; Evaluation set and canary set 15; Reading candidate outputs and choosing 10; Config, masking bugs, two learning rates 8; The GPU runs themselves 1. The GPU bill for eight to fifteen runs is about forty dollars. Everything else on this chart is paid in somebody’s attention, and the evaluation set has to come first."}},"blocks":[{"t":"h2","id":"the-gpu-bill-is-the-small-number","runs":[{"text":"The GPU bill is the small number"}]},{"t":"p","runs":[{"text":"Ask somebody what a fine-tune costs and they will quote the training run: four dollars, eleven dollars, a free Colab session. That number is real and it is almost irrelevant. What you are buying is not a run, it is an artefact with a maintenance schedule attached."}]},{"t":"p","runs":[{"text":"Here is the fuller bill for one modest production fine-tune, with the parts nobody quotes."}]},{"t":"p","runs":[{"text":"One-off, before you ship","b":true}]},{"t":"ul","items":[[{"text":"Dataset construction: 2,000 examples, of which maybe 300 are hand-written or hand-corrected. At five minutes each that is 25 hours, and that is the optimistic version."}],[{"text":"Evaluation set construction: 200 items, labelled, held out, plus a canary set for general capability. Another 15 hours, and it must come first."}],[{"text":"The training runs themselves: not one, but eight to fifteen, because you will change the template, find a masking bug, try two learning rates and two ranks. Call it $40 of GPU rather than $4."}],[{"text":"Evaluation of each candidate: cheap per run, expensive in attention."}]]},{"t":"p","runs":[{"text":"Recurring, for as long as it lives","b":true}]},{"t":"ul","items":[[{"text":"Serving. Either a card that idles at 3 a.m. or a per-token bill from a provider that hosts adapters. A single always-on L4 is roughly $450 a month; a serverless endpoint is cheaper at low volume and has cold starts you will be asked about."}],[{"text":"Monitoring. Someone has to look at the outputs, because a fine-tuned model degrades silently as the input distribution moves."}],[{"text":"Retraining. Which brings us to the real cost."}]]},{"t":"h2","id":"the-base-model-moves-under-you","runs":[{"text":"The base model moves under you"}]},{"t":"p","runs":[{"text":"A LoRA adapter is a set of weight deltas for one specific set of base weights. It does not transfer. When the base model you trained against is superseded, and open-weight families have historically shipped a meaningful upgrade every six to nine months, you have three choices, all of them costs:"}]},{"t":"ol","start":1,"items":[[{"text":"Stay on the old base.","b":true},{"text":" Your product’s quality floor is now frozen at the level of a model from last year, while the free alternative everyone else prompts keeps improving."}],[{"text":"Retrain onto the new base.","b":true},{"text":" Cheap if your dataset and pipeline are intact, expensive if they are a notebook on a laptop belonging to somebody who has left."}],[{"text":"Re-evaluate whether you still need the fine-tune at all.","b":true},{"text":" Frequently the honest answer is no: the new base does with a prompt what the old base needed training for. This happens often enough that it should be a scheduled question, not a surprise."}]]},{"t":"p","runs":[{"text":"The rule that falls out: keep the dataset, the config, and the evaluation set in version control as first-class artefacts, because the model is the "},{"text":"output","i":true},{"text":" of those things and will be regenerated several times. A team that can rebuild its fine-tune in an afternoon treats a base model upgrade as an improvement. A team that cannot treats it as a threat."}]},{"t":"h2","id":"the-second-quieter-cost","runs":[{"text":"The second, quieter cost"}]},{"t":"p","runs":[{"text":"A fine-tuned model is a dependency your organisation does not know how to reason about. Ask three practical questions about the one you are proposing:"}]},{"t":"ul","items":[[{"text":"If the person who trained it leaves next month, can somebody else reproduce it from what is written down?"}],[{"text":"When a customer complains about one specific output, what is the debugging procedure? For a prompt, you read the prompt. For weights, you look at the dataset and guess."}],[{"text":"When legal asks what the model was trained on, is there an answer?"}]]},{"t":"p","runs":[{"text":"If the answers are no, nobody knows, and not really, the fine-tune is not an asset, it is a liability that currently performs well."}]},{"t":"h2","id":"where-the-sums-work","runs":[{"text":"Where the sums work"}]},{"t":"p","runs":[{"text":"None of this is an argument against fine-tuning. It is an argument for computing the right number. The cases where the full cost is clearly worth it share a shape:"}]},{"t":"ul","items":[[{"text":"High volume.","b":true},{"text":" The saving is per-request, so the case strengthens linearly with traffic and the fixed costs amortise. Below roughly a hundred thousand requests a month it is hard to make the arithmetic work on cost alone."}],[{"text":"A stable task.","b":true},{"text":" Something whose definition will not change next quarter: transcript formatting, a fixed extraction schema, one language pair."}],[{"text":"A constraint no prompt satisfies.","b":true},{"text":" Latency budgets under 200 ms, on-device deployment, an air-gapped environment, or data that may not leave the building."}]]},{"t":"p","runs":[{"text":"Write the monthly saving and the annual maintenance on the same line before you start. If the saving is $900 a month and the maintenance is a day of engineering a month, you have not saved anything; you have converted a bill into a chore."}]}],"check":{"question":[{"text":"A team’s LoRA adapter for Llama-3.1-8B performs well. Nine months later a markedly better base model of the same family ships. Why can they not simply load their adapter onto the new base?"}],"options":[{"text":[{"text":"The adapter encodes deltas against one exact set of base weights, which different weights no longer match."}],"correct":true,"tempting":null},{"text":[{"text":"Adapters are stored in a format tied to the library version, so an older PEFT checkpoint cannot be read by newer code."}],"correct":false,"tempting":[{"text":"Confuses a format-compatibility annoyance, which is usually fixable, with the substantive reason the weights are meaningless on a different base."}]},{"text":[{"text":"The licence of the new base forbids loading adapters trained against earlier releases."}],"correct":false,"tempting":[{"text":"Invents a licence term; the obstacle is mechanical, not legal, and would exist between two permissively licensed models."}]},{"text":[{"text":"Adapters only work with the tokenizer they were trained with, and a new model release always ships a new tokenizer."}],"correct":false,"tempting":[{"text":"Picks a genuine hazard that is neither necessary nor sufficient here; tokenizers are often unchanged within a family and the adapter would still not transfer."}]}],"order":[3,1,0,2],"why":null},"prev":{"slug":"distillation-as-the-real-case","title":"Distillation, the case that usually survives","minutes":10},"next":{"slug":"licences-and-what-you-may-train-on","title":"Licences, and what you may train on","minutes":10}}}