The Honest Answer First

Start with the honest answer

Most projects that reach for fine-tuning are solved by a better prompt or by retrieval. Not some. Most. This is the first lesson because if the course changes your mind here, it has done its job and you can stop reading.

Fine-tuning has a slow loop. A prompt change takes ten seconds and you see the result immediately. A fine-tune takes a dataset, a training run, and an evaluation — a day if you are practised, a week if you are not — and when the result is worse, you often cannot tell which of the twenty things you changed did it. Anything you can learn from a prompt, learn from a prompt.

The ladder

Work down this list. Stop as soon as the problem is solved.

1. 1Write the instruction properly. State the output shape. State the constraints. Most "the model won't follow my format" complaints are a prompt that never specified the format precisely.
2. 2Put three to five real examples in the prompt. Few-shot moves format and tone further than people expect, and you can change it at 3am without a GPU.
3. 3Give it the facts. If the answer depends on your documents, policies, or prices, put the relevant text in the context window. This is the answer to "it doesn't know our stuff" nearly every time.
4. 4Give it tools. If the answer depends on a calculation or a live lookup, let it call something.
5. 5Try a stronger base model with everything above, and compare cost honestly.
6. 6Now consider fine-tuning.

The twenty-example test

Before you build anything, do this. Take twenty real inputs your system got wrong — real ones, from logs, not ones you invented. Hand-write the output you wanted for each. Put five into the prompt as examples and run the other fifteen.

• If it now gets most of the fifteen right, you do not need to fine-tune. You need that prompt.
• If it fails because it does not know a fact, you need retrieval.
• If it fails because it cannot do the reasoning at all, fine-tuning is unlikely to rescue it either.
• If it gets them right only when the examples are in the prompt, and your prompt has no room for them at your volume or your latency budget, that is a real fine-tuning case.

Those twenty hand-written outputs are also the start of your dataset, so the test is not wasted work either way.

When fine-tuning is genuinely right

• A behaviour you cannot prompt reliably at scale. A strict output shape, a house register, a domain convention that takes 2,000 tokens of instructions to describe. Fine-tuning moves that into the weights and your prompt gets short.
• Cost and latency. A tuned 7B that matches a large model on your one narrow task can be ten to fifty times cheaper per token, and faster. At a million requests a month that is the whole business case.
• Privacy or air-gap. The model has to run on your hardware, so the large hosted one is not an option at any quality.
• A narrow task with plenty of labelled data. Extraction, classification, routing. Here a small tuned model beats a large prompted one on accuracy and cost at the same time.

When it is the wrong answer

• Adding facts, especially facts that change. Use retrieval.
• Fixing hallucination. Training on answers the model does not know makes it more confidently wrong, which the next lesson explains.
• Teaching reasoning the base cannot do.
• Anything where you have forty examples, no evaluation set, and a deadline.

The rest of this course assumes you walked the ladder and still have a real case. That is a respectable place to be. It is just a much smaller group than the people currently opening a fine-tuning notebook.