Hugging Face, End to End · Chapter 07
Three cheaper things first
2:08 · Narrated with a synthetic voice. Captions are available in the player.
Read the transcript
Fine-tuning is the most satisfying answer and usually not the first correct one. Three cheaper things come before it. Fix the prompt, which very often means you had a base checkpoint or no chat template. Try a larger model, which is an afternoon rather than a weekend. And if the gap is made of facts, your prices, your policies, this year’s rules, then the answer is retrieval and not training, because facts trained into weights come back fluent, confident and subtly wrong, and cannot be updated without another run. The sentence worth memorising is that fine-tuning teaches behaviour while retrieval supplies knowledge. Most people arrive wanting the second and reach for the first. When behaviour genuinely is the gap, a format or a register you cannot express in a prompt, then it is time to train. And training a seven billion parameter model is within reach of a free session. Full fine-tuning holds the weights, their gradients and two optimiser states for every parameter, which is around fifty six gigabytes before anything else. LoRA freezes the base and learns a small low rank update instead, so you train well under one per cent of the parameters and the optimiser state shrinks by the same factor. QLoRA then keeps that frozen base in four bits, and the whole run fits in roughly seven gigabytes. One habit before you start a long run: train on ten examples for many epochs and check the model reproduces them. If it cannot memorise ten rows, your pipeline is broken rather than your data, and ten minutes has just saved you three hours.