Three cheaper things to try first

The question people skip

Fine-tuning is the most satisfying answer and usually not the first correct one. Before you spend a weekend on it, work through three cheaper options, in this order. Each of them fixes a different problem, and fine-tuning fixes only the third.

1. The prompt, and the right model

A surprising share of “the model cannot do this” turns out to be a base checkpoint used as a chat model, a missing chat template, or a prompt that never actually stated the constraint. Module two’s checklist takes ten minutes and resolves this more often than anybody expects.

Then: few-shot examples. Three worked examples in the prompt frequently close most of the gap to a fine-tune for formatting and style tasks, cost nothing but context, and can be changed in a second rather than a weekend. If your task is “produce output shaped like this”, try five examples before you try training.

And try a bigger model. An 8B model with a good prompt regularly beats a 1B model you have fine-tuned, at a fraction of the effort. Fine-tuning a small model to catch up with a large one is real work; renting the large one for a week to find out whether the task is even solvable is an afternoon.

2. Retrieval

If the gap is knowledge (your product’s documentation, your policies, this year’s prices) fine-tuning is the wrong tool and everybody learns this the hard way.

Training facts into weights is inefficient, unreliable and impossible to update. The model will produce fluent, confident, subtly wrong versions of the facts you trained on, because it learned the shape of your documents rather than their content. Retrieval puts the actual text in front of the model at the time it answers, it can be updated by adding a file, and it can cite its source.

The distinction worth memorising: fine-tuning teaches behaviour, retrieval supplies knowledge. A model that writes in your house style is fine-tuning. A model that knows your refund policy is retrieval. Most people arrive wanting the second and reach for the first.

3. A smaller, specialised model

For classification, extraction and tagging, fine-tuning a 110M encoder on a few thousand labelled examples is cheap, fast and usually more accurate than prompting a large generative model. That is fine-tuning, and it is the easy kind: an hour on a free GPU, a model that runs on a CPU afterwards, and a metric you can measure exactly.

When people say fine-tuning is hard, they mean adapting large generative models. Fine-tuning a small classifier has been routine since 2019 and remains the correct answer to a large class of problems.

When fine-tuning is the answer

• A format or style you cannot express in a prompt, or can express but not reliably enough.
• A domain vocabulary the model handles poorly: legal, clinical, a specific industry’s shorthand.
• A language or dialect the base model is weak in.
• Cost: a fine-tuned 1B model matching a prompted 70B model on one narrow task is a very large saving if you run it a million times.
• Latency: a shorter prompt, because the instructions are in the weights.
• Privacy: the task must run on your own hardware, and only a small model fits.

Notice what these have in common. They are all about how the model behaves, not what it knows.

What it costs

• Data. A few hundred high-quality examples for style, a few thousand for a real capability. Making them is most of the work and there is no way around it.
• Compute. A LoRA on a 7B model over a few thousand examples is one to four hours on a free Colab or Kaggle GPU. A few dollars if rented. Cheaper than people think.
• Evaluation. You need a held-out set before you start, or you cannot tell whether it worked.
• Maintenance. A fine-tune is pinned to its base model. When a better base appears in four months, you redo it. This is the cost nobody budgets.

The decision, in one paragraph

Try the prompt. Try a bigger model. If the gap is knowledge, build retrieval. If the task is classification, fine-tune a small encoder. If the gap is behaviour on a generative task and the first three did not close it, and you have or can make several hundred good examples, then fine-tune, and the rest of this module is how.

Do this now

Write down what you want the model to do differently, then label it: knowledge, behaviour, or format. If it is knowledge, close this module and go back to the retrieval one. That single sentence saves more time than any technique here.