Fine-tuning: what it changes and what it cannot

The decision before the technique

A team wants the model to know their product catalogue, so they fine-tune on the catalogue. It goes badly: the model produces text in catalogue style, invents plausible product codes, and is harder to correct than before.

The reason is in the objective. Fine-tuning continues the same next-token prediction on new examples. It adjusts the distribution over how text continues. It is therefore effective at:

• Format and structure. Always returning a particular shape.
• Tone and register. House voice, brevity, a specific way of declining.
• Task behaviour. A narrow classification or extraction done consistently.
• A domain’s language. Vocabulary and phrasing of a specialism.

And ineffective at:

• Adding facts reliably. Facts arrive through many thousands of correlated documents during pretraining. A few hundred examples do not install them; they teach the model to produce text shaped like your facts.
• Anything that changes. Retraining is the only update path, and it is slow.
• Anything requiring citations or revocation. The weights cannot tell you where something came from, and a deleted document is still in them.

The rule that holds up: retrieval for what the model should know, fine-tuning for how it should behave. When both are needed, do both.

LoRA, and why it made this accessible

Full fine-tuning needs the 16-bytes-per-parameter budget from earlier, impossible for most people. Low-rank adaptation avoids it.

The insight: the change a fine-tune makes to a weight matrix is empirically low-rank, so instead of updating a 4,096 × 4,096 matrix, learn two thin matrices (4,096 × 16 and 16 × 4,096) whose product is added to the original. The base weights are frozen. You are training around 0.1 to 1 per cent of the parameters.

QLoRA goes further: hold the frozen base model in 4-bit while training the adapters in higher precision. This is what puts fine-tuning a 7B model on a single consumer GPU, and it is why a free Colab or Kaggle notebook is a sufficient environment for this work.

Free tooling: Hugging Face peft and trl, axolotl, and Unsloth, which is notably faster on a single GPU. None costs anything, and Kaggle provides around 30 GPU-hours a week free.

An adapter is also small (tens of megabytes) so you can keep many for different tasks and swap them against one loaded base model. That operational property is often worth as much as the memory saving.

The settings that matter

Most tuning advice is noise. Four things carry the result:

• Data quality, overwhelmingly. A thousand carefully checked examples beat fifty thousand scraped ones. This is the whole game and everything else is rounding.
• Learning rate. 1e-4 to 2e-4 is a common LoRA range. Too high produces confident nonsense.
• Epochs. Two to three. More and it memorises; you will see validation loss rise while training loss falls.
• Rank. 8 to 32 covers most cases. Higher rank helps for new behaviour and mostly adds overfitting risk for style.

Catastrophic forgetting is real

Training on a narrow set degrades unrelated abilities. Fine-tune on customer support transcripts and general reasoning, multilingual ability and instruction-following all decline measurably.

LoRA reduces this because the base weights are untouched, but it does not eliminate it: the adapter still shifts behaviour globally. Mitigations: mix 5 to 20 per cent general instruction data into your set, keep the run short, and always evaluate on capabilities you are not training, not just on your task. A fine-tune that improves your metric by 8 points while destroying the model’s ability to say “I don’t know” is a net loss you will discover in production.

The order to try things

1. Prompt better. Clear instructions, a few good examples.
2. Add retrieval if the gap is knowledge.
3. Optimise the prompt systematically against an evaluation set.
4. Fine-tune if the gap is consistent form or behaviour that prompting cannot hold.

Most teams that reach for fine-tuning first end up back at step 1 with a more expensive setup. The one clear exception: when you need a much smaller, cheaper model to match a large one’s behaviour on a narrow task, fine-tuning on the large model’s outputs is the right tool and often works well.