When not to fine-tune

What fine-tuning changes, and what it does not

Fine-tuning adjusts weights on examples of the behaviour you want. It is very good at some things and unreliable at others, and the difference decides whether you should do it.

It reliably changes form. Output format, tone, register, verbosity, structure, the conventions of your domain, which of several plausible responses is preferred, and whether the model does the task at all rather than explaining it. This is what fine-tuning is for.

It changes knowledge poorly. You can teach facts by fine-tuning, and it works badly compared with putting the facts in the context. The model learns the surface of the statement, generalises it unpredictably, contradicts it when phrased differently, and forgets other things in the process. Retrieval is better on every axis: cheaper, updatable, citable, and it does not degrade anything else.

It does not add reasoning capacity. A 3B model tuned on your data is still a 3B model. Tuning can make it apply what it has more reliably to your task; it cannot make it think further ahead.

The ladder, in order of cost

Work down this list and stop when the result is good enough.

1. A better prompt. An hour, and it fixes more than people expect.
2. Three to eight examples in the prompt. Another hour.
3. Constrained output. Removes format failures entirely.
4. Retrieval. Fixes knowledge problems, which is most of what looks like model weakness.
5. A different or larger model. A day of evaluation.
6. Fine-tuning. Days of data work, then permanent maintenance.

Most teams that fine-tune skipped steps 3 and 4. That is the single most common waste in this part of the field.

The signals that fine-tuning is right

• You have, or can get, 300 or more real examples of exactly the input and the output you want. Below that, few-shot prompting is usually as good.
• A long prompt is being paid on every request. If you are sending 2,000 tokens of instructions and examples a hundred thousand times a month, that is 200 million input tokens. Tuning that behaviour into the weights removes the recurring cost, and this is often the strongest financial argument.
• You need a smaller model than the task needs. A tuned 3B that matches an untuned 30B on your narrow task costs an order of magnitude less to serve and runs where the 30B cannot. This is the main reason fine-tuning is worth it on device.
• A format or style must hold over long outputs. Prompted constraints decay; trained behaviour does not.
• The model is systematically wrong in a way you can demonstrate, on a domain with its own conventions, and you have examples of right.

The cost people underestimate

The run is the cheap part. A LoRA on 500 examples is under an hour on a free notebook GPU.

The expensive parts are the dataset, which is two or three days of careful work by somebody who understands the task, and the permanence. A fine-tune is a fork. When a better base model appears in four months, you re-tune, re-evaluate and re-deploy, or you stay on the old base while the field moves. Every fine-tune is a recurring commitment, and teams routinely take on three or four of them before noticing.

Budget it honestly: the first tune costs days, and each subsequent base upgrade costs a day. If the improvement does not justify that annually, do not start.

A decision you can defend

Write down, before you begin, three things:

1. The number your fine-tune must beat, measured on a held-out set with the best prompt you could write.
2. The margin that would count as worth it, given the noise on a set that size.
3. What you will do when the base model is superseded.

If you cannot fill in the third line, you are not ready to fine-tune. That is not a rhetorical flourish; abandoned fine-tunes running on two-year-old bases are one of the most common forms of technical debt in this work.

The most useful sentence in this lesson, if you take only one: measure the best prompted result first, write it down, and treat it as the thing to beat. Half the time nothing beats it and you have saved a week.