{"course":{"slug":"hugging-face","title":"Hugging Face, End to End","url":"https://addaly.com/learn/hugging-face","level":"intermediate","track":"create"},"lesson":{"slug":"when-not-to-fine-tune","title":"Three cheaper things to try first","url":"https://addaly.com/learn/hugging-face/when-not-to-fine-tune","position":57,"of":75,"minutes":8,"answer":"Fine-tuning teaches behaviour and retrieval supplies knowledge, so a gap made of facts should never be trained into weights, and prompting.","keyIdea":"Fine-tuning teaches behaviour and retrieval supplies knowledge, so a gap made of facts should never be trained into weights, and prompting, a larger model and a small specialised classifier all sit between the problem and a training run that will need redoing when the base model is replaced."},"body":{"format":"markdown","text":"## The question people skip\n\nFine-tuning is the most satisfying answer and usually not the first correct one. Before you spend a weekend on it, work through three cheaper options, in this order. Each of them fixes a different problem, and fine-tuning fixes only the third.\n\n## 1. The prompt, and the right model\n\nA surprising share of \"the model cannot do this\" turns out to be a base checkpoint used as a chat model, a missing chat template, or a prompt that never actually stated the constraint. Module two's checklist takes ten minutes and resolves this more often than anybody expects.\n\nThen: few-shot examples. Three worked examples in the prompt frequently close most of the gap to a fine-tune for formatting and style tasks, cost nothing but context, and can be changed in a second rather than a weekend. If your task is \"produce output shaped like this\", try five examples before you try training.\n\n> Figure: Three cheaper things, in order. Steps: 1. Fix the prompt (Instruct model, template); 2. Try a bigger model (An afternoon, not a weekend); 3. Gap made of facts? (Retrieval, not weights); 4. A classifier? (A 110M encoder, one hour); 5. Now fine-tune (Behaviour, format, register). Each step fixes a different problem and fine-tuning fixes only the last. The distinction worth memorising sits in the middle: fine-tuning teaches behaviour and retrieval supplies knowledge, so a gap made of facts should never be trained into weights.\n\nAnd try a bigger model. An 8B model with a good prompt regularly beats a 1B model you have fine-tuned, at a fraction of the effort. Fine-tuning a small model to catch up with a large one is real work; renting the large one for a week to find out whether the task is even solvable is an afternoon.\n\n## 2. Retrieval\n\nIf the gap is **knowledge** (your product's documentation, your policies, this year's prices) fine-tuning is the wrong tool and everybody learns this the hard way.\n\nTraining facts into weights is inefficient, unreliable and impossible to update. The model will produce fluent, confident, subtly wrong versions of the facts you trained on, because it learned the *shape* of your documents rather than their content. Retrieval puts the actual text in front of the model at the time it answers, it can be updated by adding a file, and it can cite its source.\n\nThe distinction worth memorising: **fine-tuning teaches behaviour, retrieval supplies knowledge.** A model that writes in your house style is fine-tuning. A model that knows your refund policy is retrieval. Most people arrive wanting the second and reach for the first.\n\n## 3. A smaller, specialised model\n\nFor classification, extraction and tagging, fine-tuning a 110M encoder on a few thousand labelled examples is cheap, fast and usually more accurate than prompting a large generative model. That *is* fine-tuning, and it is the easy kind: an hour on a free GPU, a model that runs on a CPU afterwards, and a metric you can measure exactly.\n\nWhen people say fine-tuning is hard, they mean adapting large generative models. Fine-tuning a small classifier has been routine since 2019 and remains the correct answer to a large class of problems.\n\n## When fine-tuning is the answer\n\n- **A format or style you cannot express in a prompt**, or can express but not reliably enough.\n- **A domain vocabulary** the model handles poorly: legal, clinical, a specific industry's shorthand.\n- **A language or dialect** the base model is weak in.\n- **Cost**: a fine-tuned 1B model matching a prompted 70B model on one narrow task is a very large saving if you run it a million times.\n- **Latency**: a shorter prompt, because the instructions are in the weights.\n- **Privacy**: the task must run on your own hardware, and only a small model fits.\n\nNotice what these have in common. They are all about **how** the model behaves, not **what** it knows.\n\n## What it costs\n\n- **Data.** A few hundred high-quality examples for style, a few thousand for a real capability. Making them is most of the work and there is no way around it.\n- **Compute.** A LoRA on a 7B model over a few thousand examples is one to four hours on a free Colab or Kaggle GPU. A few dollars if rented. Cheaper than people think.\n- **Evaluation.** You need a held-out set before you start, or you cannot tell whether it worked.\n- **Maintenance.** A fine-tune is pinned to its base model. When a better base appears in four months, you redo it. This is the cost nobody budgets.\n\n## The decision, in one paragraph\n\nTry the prompt. Try a bigger model. If the gap is knowledge, build retrieval. If the task is classification, fine-tune a small encoder. If the gap is behaviour on a generative task and the first three did not close it, and you have or can make several hundred good examples, then fine-tune, and the rest of this module is how.\n\n## Do this now\n\nWrite down what you want the model to do differently, then label it: knowledge, behaviour, or format. If it is knowledge, close this module and go back to the retrieval one. That single sentence saves more time than any technique here."},"check":{"question":"A team fine-tunes a 7B model on their product documentation so it can answer customer questions. It becomes fluent about the product and gets specific details wrong. Why?","options":[{"text":"The learning rate was too low to memorise facts, so more epochs would fix the detail errors.","correct":false,"tempting":"More training on the same data increases memorisation of phrasing rather than reliable factual recall."},{"text":"Training taught the shape of the documents rather than facts, which retrieval supplies directly.","correct":true},{"text":"The documentation was too small a dataset, and tens of thousands of examples would have made the facts stick.","correct":false,"tempting":"Scaling the same approach makes the model more confident without making updates possible or recall exact."},{"text":"The base model's pretraining contradicts the documentation, so the two sources of knowledge interfere with one another.","correct":false,"tempting":"Interference is real but secondary to the fact that weights are a poor store for facts that must be exact and updatable."}]},"prev":{"slug":"measuring-your-retrieval","title":"Thirty questions, and the numbers they give you","url":"https://addaly.com/learn/hugging-face/measuring-your-retrieval"},"next":{"slug":"the-trainer-api","title":"Trainer, and what it saves you writing","url":"https://addaly.com/learn/hugging-face/the-trainer-api"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Three cheaper things to try first', in Hugging Face, End to End. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/hugging-face/when-not-to-fine-tune (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Three cheaper things to try first","courseTitle":"Hugging Face, End to End","module":{"index":7,"slug":"training-with-the-stack","title":"Training and adapting, with the libraries the Hub is built on"},"answer":null,"keyIdea":"Fine-tuning teaches behaviour and retrieval supplies knowledge, so a gap made of facts should never be trained into weights, and prompting, a larger model and a small specialised classifier all sit between the problem and a training run that will need redoing when the base model is replaced.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 7.1","figure":{"type":"flow","steps":[{"label":"Fix the prompt","detail":"Instruct model, template"},{"label":"Try a bigger model","detail":"An afternoon, not a weekend"},{"label":"Gap made of facts?","detail":"Retrieval, not weights"},{"label":"A classifier?","detail":"A 110M encoder, one hour"},{"label":"Now fine-tune","detail":"Behaviour, format, register"}],"title":"Three cheaper things, in order","caption":"Each step fixes a different problem and fine-tuning fixes only the last. The distinction worth memorising sits in the middle: fine-tuning teaches behaviour and retrieval supplies knowledge, so a gap made of facts should never be trained into weights."},"alt":"Figure: Three cheaper things, in order. Steps: 1. Fix the prompt (Instruct model, template); 2. Try a bigger model (An afternoon, not a weekend); 3. Gap made of facts? (Retrieval, not weights); 4. A classifier? (A 110M encoder, one hour); 5. Now fine-tune (Behaviour, format, register). Each step fixes a different problem and fine-tuning fixes only the last. The distinction worth memorising sits in the middle: fine-tuning teaches behaviour and retrieval supplies knowledge, so a gap made of facts should never be trained into weights."}},"blocks":[{"t":"h2","id":"the-question-people-skip","runs":[{"text":"The question people skip"}]},{"t":"p","runs":[{"text":"Fine-tuning is the most satisfying answer and usually not the first correct one. Before you spend a weekend on it, work through three cheaper options, in this order. Each of them fixes a different problem, and fine-tuning fixes only the third."}]},{"t":"h2","id":"1-the-prompt-and-the-right-model","runs":[{"text":"1. The prompt, and the right model"}]},{"t":"p","runs":[{"text":"A surprising share of “the model cannot do this” turns out to be a base checkpoint used as a chat model, a missing chat template, or a prompt that never actually stated the constraint. Module two’s checklist takes ten minutes and resolves this more often than anybody expects."}]},{"t":"p","runs":[{"text":"Then: few-shot examples. Three worked examples in the prompt frequently close most of the gap to a fine-tune for formatting and style tasks, cost nothing but context, and can be changed in a second rather than a weekend. If your task is “produce output shaped like this”, try five examples before you try training."}]},{"t":"p","runs":[{"text":"And try a bigger model. An 8B model with a good prompt regularly beats a 1B model you have fine-tuned, at a fraction of the effort. Fine-tuning a small model to catch up with a large one is real work; renting the large one for a week to find out whether the task is even solvable is an afternoon."}]},{"t":"h2","id":"2-retrieval","runs":[{"text":"2. Retrieval"}]},{"t":"p","runs":[{"text":"If the gap is "},{"text":"knowledge","b":true},{"text":" (your product’s documentation, your policies, this year’s prices) fine-tuning is the wrong tool and everybody learns this the hard way."}]},{"t":"p","runs":[{"text":"Training facts into weights is inefficient, unreliable and impossible to update. The model will produce fluent, confident, subtly wrong versions of the facts you trained on, because it learned the "},{"text":"shape","i":true},{"text":" of your documents rather than their content. Retrieval puts the actual text in front of the model at the time it answers, it can be updated by adding a file, and it can cite its source."}]},{"t":"p","runs":[{"text":"The distinction worth memorising: "},{"text":"fine-tuning teaches behaviour, retrieval supplies knowledge.","b":true},{"text":" A model that writes in your house style is fine-tuning. A model that knows your refund policy is retrieval. Most people arrive wanting the second and reach for the first."}]},{"t":"h2","id":"3-a-smaller-specialised-model","runs":[{"text":"3. A smaller, specialised model"}]},{"t":"p","runs":[{"text":"For classification, extraction and tagging, fine-tuning a 110M encoder on a few thousand labelled examples is cheap, fast and usually more accurate than prompting a large generative model. That "},{"text":"is","i":true},{"text":" fine-tuning, and it is the easy kind: an hour on a free GPU, a model that runs on a CPU afterwards, and a metric you can measure exactly."}]},{"t":"p","runs":[{"text":"When people say fine-tuning is hard, they mean adapting large generative models. Fine-tuning a small classifier has been routine since 2019 and remains the correct answer to a large class of problems."}]},{"t":"h2","id":"when-fine-tuning-is-the-answer","runs":[{"text":"When fine-tuning is the answer"}]},{"t":"ul","items":[[{"text":"A format or style you cannot express in a prompt","b":true},{"text":", or can express but not reliably enough."}],[{"text":"A domain vocabulary","b":true},{"text":" the model handles poorly: legal, clinical, a specific industry’s shorthand."}],[{"text":"A language or dialect","b":true},{"text":" the base model is weak in."}],[{"text":"Cost","b":true},{"text":": a fine-tuned 1B model matching a prompted 70B model on one narrow task is a very large saving if you run it a million times."}],[{"text":"Latency","b":true},{"text":": a shorter prompt, because the instructions are in the weights."}],[{"text":"Privacy","b":true},{"text":": the task must run on your own hardware, and only a small model fits."}]]},{"t":"p","runs":[{"text":"Notice what these have in common. They are all about "},{"text":"how","b":true},{"text":" the model behaves, not "},{"text":"what","b":true},{"text":" it knows."}]},{"t":"h2","id":"what-it-costs","runs":[{"text":"What it costs"}]},{"t":"ul","items":[[{"text":"Data.","b":true},{"text":" A few hundred high-quality examples for style, a few thousand for a real capability. Making them is most of the work and there is no way around it."}],[{"text":"Compute.","b":true},{"text":" A LoRA on a 7B model over a few thousand examples is one to four hours on a free Colab or Kaggle GPU. A few dollars if rented. Cheaper than people think."}],[{"text":"Evaluation.","b":true},{"text":" You need a held-out set before you start, or you cannot tell whether it worked."}],[{"text":"Maintenance.","b":true},{"text":" A fine-tune is pinned to its base model. When a better base appears in four months, you redo it. This is the cost nobody budgets."}]]},{"t":"h2","id":"the-decision-in-one-paragraph","runs":[{"text":"The decision, in one paragraph"}]},{"t":"p","runs":[{"text":"Try the prompt. Try a bigger model. If the gap is knowledge, build retrieval. If the task is classification, fine-tune a small encoder. If the gap is behaviour on a generative task and the first three did not close it, and you have or can make several hundred good examples, then fine-tune, and the rest of this module is how."}]},{"t":"h2","id":"do-this-now","runs":[{"text":"Do this now"}],"role":"try"},{"t":"p","runs":[{"text":"Write down what you want the model to do differently, then label it: knowledge, behaviour, or format. If it is knowledge, close this module and go back to the retrieval one. That single sentence saves more time than any technique here."}]}],"check":{"question":[{"text":"A team fine-tunes a 7B model on their product documentation so it can answer customer questions. It becomes fluent about the product and gets specific details wrong. Why?"}],"options":[{"text":[{"text":"The learning rate was too low to memorise facts, so more epochs would fix the detail errors."}],"correct":false,"tempting":[{"text":"More training on the same data increases memorisation of phrasing rather than reliable factual recall."}]},{"text":[{"text":"Training taught the shape of the documents rather than facts, which retrieval supplies directly."}],"correct":true,"tempting":null},{"text":[{"text":"The documentation was too small a dataset, and tens of thousands of examples would have made the facts stick."}],"correct":false,"tempting":[{"text":"Scaling the same approach makes the model more confident without making updates possible or recall exact."}]},{"text":[{"text":"The base model’s pretraining contradicts the documentation, so the two sources of knowledge interfere with one another."}],"correct":false,"tempting":[{"text":"Interference is real but secondary to the fact that weights are a poor store for facts that must be exact and updatable."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"measuring-your-retrieval","title":"Thirty questions, and the numbers they give you","minutes":9},"next":{"slug":"the-trainer-api","title":"Trainer, and what it saves you writing","minutes":9}}}