{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"fine-tuning-in-practice","title":"Fine-tuning: what it changes and what it cannot","url":"https://addaly.com/learn/how-llms-work/fine-tuning-in-practice","position":45,"of":78,"minutes":9,"answer":"Fine-tuning is good at teaching form and poor at adding facts, and confusing the two is the commonest expensive mistake in applied work.","keyIdea":"Fine-tuning is good at teaching form and poor at adding facts, and confusing the two is the commonest expensive mistake in applied work."},"body":{"format":"markdown","text":"## The decision before the technique\n\nA team wants the model to know their product catalogue, so they fine-tune on the catalogue. It goes badly: the model produces text in catalogue *style*, invents plausible product codes, and is harder to correct than before.\n\nThe reason is in the objective. Fine-tuning continues the same next-token prediction on new examples. It adjusts the distribution over how text continues. It is therefore effective at:\n\n- **Format and structure.** Always returning a particular shape.\n- **Tone and register.** House voice, brevity, a specific way of declining.\n- **Task behaviour.** A narrow classification or extraction done consistently.\n- **A domain's language.** Vocabulary and phrasing of a specialism.\n\nAnd ineffective at:\n\n- **Adding facts reliably.** Facts arrive through many thousands of correlated documents during pretraining. A few hundred examples do not install them; they teach the model to produce text shaped like your facts.\n- **Anything that changes.** Retraining is the only update path, and it is slow.\n- **Anything requiring citations or revocation.** The weights cannot tell you where something came from, and a deleted document is still in them.\n\nThe rule that holds up: **retrieval for what the model should know, fine-tuning for how it should behave.** When both are needed, do both.\n\n> Figure: Fine-tuning or retrieval: the two-way test. A two by two grid, rows Fine-tuning and Retrieval, columns Changing form and Adding facts: Fine-tuning and Changing form: works; Fine-tuning and Adding facts: fails; Retrieval and Changing form: no effect; Retrieval and Adding facts: works. The catalogue team fine-tuned to install their catalogue and got catalogue prose with invented product codes. That is the expected result rather than bad luck: a short run over a small corpus teaches a voice, because the voice is the pattern present on every page.\n\n## LoRA, and why it made this accessible\n\nFull fine-tuning needs the 16-bytes-per-parameter budget from earlier, impossible for most people. Low-rank adaptation avoids it.\n\nThe insight: the *change* a fine-tune makes to a weight matrix is empirically low-rank, so instead of updating a 4,096 × 4,096 matrix, learn two thin matrices (4,096 × 16 and 16 × 4,096) whose product is added to the original. The base weights are frozen. You are training around 0.1 to 1 per cent of the parameters.\n\n**QLoRA** goes further: hold the frozen base model in 4-bit while training the adapters in higher precision. This is what puts fine-tuning a 7B model on a single consumer GPU, and it is why a free Colab or Kaggle notebook is a sufficient environment for this work.\n\nFree tooling: Hugging Face `peft` and `trl`, `axolotl`, and Unsloth, which is notably faster on a single GPU. None costs anything, and Kaggle provides around 30 GPU-hours a week free.\n\nAn adapter is also small (tens of megabytes) so you can keep many for different tasks and swap them against one loaded base model. That operational property is often worth as much as the memory saving.\n\n## The settings that matter\n\nMost tuning advice is noise. Four things carry the result:\n\n- **Data quality, overwhelmingly.** A thousand carefully checked examples beat fifty thousand scraped ones. This is the whole game and everything else is rounding.\n- **Learning rate.** 1e-4 to 2e-4 is a common LoRA range. Too high produces confident nonsense.\n- **Epochs.** Two to three. More and it memorises; you will see validation loss rise while training loss falls.\n- **Rank.** 8 to 32 covers most cases. Higher rank helps for new behaviour and mostly adds overfitting risk for style.\n\n## Catastrophic forgetting is real\n\nTraining on a narrow set degrades unrelated abilities. Fine-tune on customer support transcripts and general reasoning, multilingual ability and instruction-following all decline measurably.\n\nLoRA reduces this because the base weights are untouched, but it does not eliminate it: the adapter still shifts behaviour globally. Mitigations: mix 5 to 20 per cent general instruction data into your set, keep the run short, and **always evaluate on capabilities you are not training**, not just on your task. A fine-tune that improves your metric by 8 points while destroying the model's ability to say \"I don't know\" is a net loss you will discover in production.\n\n## The order to try things\n\n1. **Prompt better.** Clear instructions, a few good examples.\n2. **Add retrieval** if the gap is knowledge.\n3. **Optimise the prompt systematically** against an evaluation set.\n4. **Fine-tune** if the gap is consistent form or behaviour that prompting cannot hold.\n\nMost teams that reach for fine-tuning first end up back at step 1 with a more expensive setup. The one clear exception: when you need a much smaller, cheaper model to match a large one's behaviour on a narrow task, fine-tuning on the large model's outputs is the right tool and often works well."},"check":{"question":"A company fine-tunes a model on 4,000 internal policy documents so it can answer staff questions. It now answers in convincing policy language with invented clause numbers. Why?","options":[{"text":"The learning rate was too high, causing the model to overfit the document style.","correct":false,"tempting":"Overfitting to style is a genuine risk and learning rate genuinely controls it, which makes this the most technically plausible wrong answer."},{"text":"Fine-tuning adjusts the distribution over how text continues, so it teaches the form of policy writing rather than installing retrievable facts.","correct":true},{"text":"Four thousand documents is too few; a larger corpus would install the facts reliably.","correct":false,"tempting":"Data volume does matter, and the number sounds small, so scaling up feels like the obvious remedy."},{"text":"The documents were not formatted as question-and-answer pairs, so the model learned continuation instead.","correct":false,"tempting":"Format genuinely matters for instruction tuning, which makes this a real and useful point aimed at the wrong failure."}]},"prev":{"slug":"what-training-costs","title":"What a training run actually costs","url":"https://addaly.com/learn/how-llms-work/what-training-costs"},"next":{"slug":"preference-training","title":"Preference training, from reward models to DPO","url":"https://addaly.com/learn/how-llms-work/preference-training"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Fine-tuning: what it changes and what it cannot', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/fine-tuning-in-practice (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Fine-tuning: what it changes and what it cannot","courseTitle":"How a Language Model Actually Works","module":{"index":5,"slug":"where-the-weights-come-from","title":"Where the weights come from"},"answer":null,"keyIdea":"Fine-tuning is good at teaching form and poor at adding facts, and confusing the two is the commonest expensive mistake in applied work.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"cols":["Changing form","Adding facts"],"rows":["Fine-tuning","Retrieval"],"type":"matrix","cells":[["works","fails"],["no effect","works"]],"title":"Fine-tuning or retrieval: the two-way test","caption":"The catalogue team fine-tuned to install their catalogue and got catalogue prose with invented product codes. That is the expected result rather than bad luck: a short run over a small corpus teaches a voice, because the voice is the pattern present on every page.","cellLabels":[["House format, tone, a rigid schema, one narrow task","Text shaped like your facts, with invented specifics"],["The model still writes the way it wrote","Current, private, citable, revocable the moment a document is deleted"]]},"alt":"Figure: Fine-tuning or retrieval: the two-way test. A two by two grid, rows Fine-tuning and Retrieval, columns Changing form and Adding facts: Fine-tuning and Changing form: works; Fine-tuning and Adding facts: fails; Retrieval and Changing form: no effect; Retrieval and Adding facts: works. The catalogue team fine-tuned to install their catalogue and got catalogue prose with invented product codes. That is the expected result rather than bad luck: a short run over a small corpus teaches a voice, because the voice is the pattern present on every page."}},"blocks":[{"t":"h2","id":"the-decision-before-the-technique","runs":[{"text":"The decision before the technique"}]},{"t":"p","runs":[{"text":"A team wants the model to know their product catalogue, so they fine-tune on the catalogue. It goes badly: the model produces text in catalogue "},{"text":"style","i":true},{"text":", invents plausible product codes, and is harder to correct than before."}]},{"t":"p","runs":[{"text":"The reason is in the objective. Fine-tuning continues the same next-token prediction on new examples. It adjusts the distribution over how text continues. It is therefore effective at:"}]},{"t":"ul","items":[[{"text":"Format and structure.","b":true},{"text":" Always returning a particular shape."}],[{"text":"Tone and register.","b":true},{"text":" House voice, brevity, a specific way of declining."}],[{"text":"Task behaviour.","b":true},{"text":" A narrow classification or extraction done consistently."}],[{"text":"A domain’s language.","b":true},{"text":" Vocabulary and phrasing of a specialism."}]]},{"t":"p","runs":[{"text":"And ineffective at:"}]},{"t":"ul","items":[[{"text":"Adding facts reliably.","b":true},{"text":" Facts arrive through many thousands of correlated documents during pretraining. A few hundred examples do not install them; they teach the model to produce text shaped like your facts."}],[{"text":"Anything that changes.","b":true},{"text":" Retraining is the only update path, and it is slow."}],[{"text":"Anything requiring citations or revocation.","b":true},{"text":" The weights cannot tell you where something came from, and a deleted document is still in them."}]]},{"t":"p","runs":[{"text":"The rule that holds up: "},{"text":"retrieval for what the model should know, fine-tuning for how it should behave.","b":true},{"text":" When both are needed, do both."}]},{"t":"h2","id":"lora-and-why-it-made-this-accessible","runs":[{"text":"LoRA, and why it made this accessible"}]},{"t":"p","runs":[{"text":"Full fine-tuning needs the 16-bytes-per-parameter budget from earlier, impossible for most people. Low-rank adaptation avoids it."}]},{"t":"p","runs":[{"text":"The insight: the "},{"text":"change","i":true},{"text":" a fine-tune makes to a weight matrix is empirically low-rank, so instead of updating a 4,096 × 4,096 matrix, learn two thin matrices (4,096 × 16 and 16 × 4,096) whose product is added to the original. The base weights are frozen. You are training around 0.1 to 1 per cent of the parameters."}]},{"t":"p","runs":[{"text":"QLoRA","b":true},{"text":" goes further: hold the frozen base model in 4-bit while training the adapters in higher precision. This is what puts fine-tuning a 7B model on a single consumer GPU, and it is why a free Colab or Kaggle notebook is a sufficient environment for this work."}]},{"t":"p","runs":[{"text":"Free tooling: Hugging Face "},{"text":"peft","code":true},{"text":" and "},{"text":"trl","code":true},{"text":", "},{"text":"axolotl","code":true},{"text":", and Unsloth, which is notably faster on a single GPU. None costs anything, and Kaggle provides around 30 GPU-hours a week free."}]},{"t":"p","runs":[{"text":"An adapter is also small (tens of megabytes) so you can keep many for different tasks and swap them against one loaded base model. That operational property is often worth as much as the memory saving."}]},{"t":"h2","id":"the-settings-that-matter","runs":[{"text":"The settings that matter"}]},{"t":"p","runs":[{"text":"Most tuning advice is noise. Four things carry the result:"}]},{"t":"ul","items":[[{"text":"Data quality, overwhelmingly.","b":true},{"text":" A thousand carefully checked examples beat fifty thousand scraped ones. This is the whole game and everything else is rounding."}],[{"text":"Learning rate.","b":true},{"text":" 1e-4 to 2e-4 is a common LoRA range. Too high produces confident nonsense."}],[{"text":"Epochs.","b":true},{"text":" Two to three. More and it memorises; you will see validation loss rise while training loss falls."}],[{"text":"Rank.","b":true},{"text":" 8 to 32 covers most cases. Higher rank helps for new behaviour and mostly adds overfitting risk for style."}]]},{"t":"h2","id":"catastrophic-forgetting-is-real","runs":[{"text":"Catastrophic forgetting is real"}]},{"t":"p","runs":[{"text":"Training on a narrow set degrades unrelated abilities. Fine-tune on customer support transcripts and general reasoning, multilingual ability and instruction-following all decline measurably."}]},{"t":"p","runs":[{"text":"LoRA reduces this because the base weights are untouched, but it does not eliminate it: the adapter still shifts behaviour globally. Mitigations: mix 5 to 20 per cent general instruction data into your set, keep the run short, and "},{"text":"always evaluate on capabilities you are not training","b":true},{"text":", not just on your task. A fine-tune that improves your metric by 8 points while destroying the model’s ability to say “I don’t know” is a net loss you will discover in production."}]},{"t":"h2","id":"the-order-to-try-things","runs":[{"text":"The order to try things"}]},{"t":"ol","start":1,"items":[[{"text":"Prompt better.","b":true},{"text":" Clear instructions, a few good examples."}],[{"text":"Add retrieval","b":true},{"text":" if the gap is knowledge."}],[{"text":"Optimise the prompt systematically","b":true},{"text":" against an evaluation set."}],[{"text":"Fine-tune","b":true},{"text":" if the gap is consistent form or behaviour that prompting cannot hold."}]]},{"t":"p","runs":[{"text":"Most teams that reach for fine-tuning first end up back at step 1 with a more expensive setup. The one clear exception: when you need a much smaller, cheaper model to match a large one’s behaviour on a narrow task, fine-tuning on the large model’s outputs is the right tool and often works well."}]}],"check":{"question":[{"text":"A company fine-tunes a model on 4,000 internal policy documents so it can answer staff questions. It now answers in convincing policy language with invented clause numbers. Why?"}],"options":[{"text":[{"text":"The learning rate was too high, causing the model to overfit the document style."}],"correct":false,"tempting":[{"text":"Overfitting to style is a genuine risk and learning rate genuinely controls it, which makes this the most technically plausible wrong answer."}]},{"text":[{"text":"Fine-tuning adjusts the distribution over how text continues, so it teaches the form of policy writing rather than installing retrievable facts."}],"correct":true,"tempting":null},{"text":[{"text":"Four thousand documents is too few; a larger corpus would install the facts reliably."}],"correct":false,"tempting":[{"text":"Data volume does matter, and the number sounds small, so scaling up feels like the obvious remedy."}]},{"text":[{"text":"The documents were not formatted as question-and-answer pairs, so the model learned continuation instead."}],"correct":false,"tempting":[{"text":"Format genuinely matters for instruction tuning, which makes this a real and useful point aimed at the wrong failure."}]}],"order":[3,1,0,2],"why":null},"prev":{"slug":"what-training-costs","title":"What a training run actually costs","minutes":8},"next":{"slug":"preference-training","title":"Preference training, from reward models to DPO","minutes":9}}}