{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"running-your-first-fine-tune","title":"A real run, on free hardware","url":"https://addaly.com/learn/open-models/running-your-first-fine-tune","position":55,"of":71,"minutes":10,"answer":"A LoRA run is thirty lines and an hour on free hardware.","keyIdea":"A LoRA run is thirty lines and an hour on free hardware, but the loss curve is only diagnostics: the result is the task metric on a held-out set compared against the base model with your best prompt, at the same settings."},"body":{"format":"markdown","text":"## The tools\n\n**Unsloth**: patched kernels giving roughly twice the speed and noticeably less memory than the baseline, with free notebooks for every popular base model. The fastest path from nothing to a trained adapter.\n\n**TRL**: Hugging Face's training library. `SFTTrainer` is the standard implementation and what most other tools wrap.\n\n**Axolotl**: configuration by YAML rather than code. Better for reproducibility and for running many variants.\n\n**LLaMA-Factory**: a graphical interface over the same ideas, if you would rather not write the script.\n\nAll four do the same thing. Pick one and learn its defaults.\n\n## A complete run\n\n```python\nfrom datasets import load_dataset\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n\nbase = \"Qwen/Qwen3-8B\"\ntok  = AutoTokenizer.from_pretrained(base)\nmodel = AutoModelForCausalLM.from_pretrained(\n    base, device_map=\"auto\", torch_dtype=\"bfloat16\",\n    quantization_config=BitsAndBytesConfig(load_in_4bit=True,\n        bnb_4bit_quant_type=\"nf4\", bnb_4bit_compute_dtype=\"bfloat16\"),\n)\n\nds = load_dataset(\"json\", data_files={\"train\":\"train.jsonl\",\"test\":\"dev.jsonl\"})\n\ntrainer = SFTTrainer(\n    model=model, tokenizer=tok,\n    train_dataset=ds[\"train\"], eval_dataset=ds[\"test\"],\n    peft_config=LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,\n        target_modules=[\"q_proj\",\"k_proj\",\"v_proj\",\"o_proj\",\n                        \"gate_proj\",\"up_proj\",\"down_proj\"]),\n    args=SFTConfig(\n        output_dir=\"out\", max_seq_length=1024, packing=True,\n        per_device_train_batch_size=1, gradient_accumulation_steps=16,\n        gradient_checkpointing=True, num_train_epochs=2,\n        learning_rate=2e-4, lr_scheduler_type=\"cosine\", warmup_ratio=0.03,\n        logging_steps=10, eval_strategy=\"steps\", eval_steps=50,\n        save_steps=50, save_total_limit=2, bf16=True, seed=42,\n    ),\n)\ntrainer.train()\ntrainer.model.save_pretrained(\"out/adapter\")\n```\n\nOn 500 examples of about 600 tokens each, two epochs, this takes 30 to 60 minutes on a free notebook GPU.\n\n## The four hyperparameters that matter\n\n**Learning rate.** For LoRA, `1e-4` to `2e-4`. Full fine-tuning wants roughly ten times lower, around `1e-5` to `2e-5`, and using a LoRA learning rate for a full tune destroys the model. If the loss goes to zero in fifty steps or turns to `nan`, the rate is too high.\n\n**Epochs.** One to three on a few hundred examples. Small datasets overfit quickly, and the third epoch is often where the model stops generalising and starts reciting.\n\n**Effective batch size.** Batch size times accumulation steps. Sixteen to thirty-two is a reasonable range. Too small makes training noisy; too large wastes the small dataset.\n\n**Max sequence length.** Set it to your real data with `packing=True`, which fills each sequence with several examples rather than padding. On short examples packing can double throughput.\n\nEverything else can stay at defaults for a first run.\n\n## Reading the curves honestly\n\n**Training loss falling** means the model is fitting your examples. It says nothing about whether it does your task better.\n\n**Evaluation loss** is more useful, and the shape to watch for is a fall then a rise: the point where it turns upward is where overfitting begins, and the checkpoint before it is the one you want.\n\n**Neither number is your metric.** Loss is average token surprise. Your task is accuracy, or valid JSON rate, or whatever the product needs. A model can have lower loss and be worse for you: for instance because it learned to reproduce a verbose style that raises token-level agreement while making the extraction harder to parse.\n\nAlways run the actual task evaluation on the held-out set, at temperature 0, against the base model with your best prompt. Two numbers, same set, same settings. That comparison is the result; the loss curve is diagnostics.\n\n## What a failed run looks like\n\n- **Loss barely moves.** Wrong target modules, learning rate far too low, or the loss is being computed on the wrong tokens. Print the trainable parameter count first.\n- **Loss collapses to near zero in a few steps.** Learning rate too high, or the dataset has near-duplicates being memorised.\n- **The model outputs the training format for every input, including irrelevant ones.** Overfitting. Fewer epochs, lower rank, more diverse data.\n- **Output is fluent nonsense.** The chat template used in training does not match the one used at inference. Check both.\n\nThat last one is the most common of all, and the fix is to generate with the exact same template the trainer applied."},"check":{"question":"After fine-tuning, evaluation loss is clearly lower than the base model's but task accuracy on the held-out set is slightly worse. What is the most reasonable interpretation?","options":[{"text":"The evaluation set is too small, so the accuracy difference is noise and the loss should be trusted.","correct":false,"tempting":"Noise is worth checking, but loss and task accuracy measure different things, so agreement between them was never guaranteed."},{"text":"The learning rate was too high, which lowers loss while damaging the model.","correct":false,"tempting":"A rate that damages the model raises loss rather than lowering it."},{"text":"Loss measures token-level agreement with your training style, which the model learned, while accuracy measures the task, so the two can move in opposite directions.","correct":true},{"text":"The model has overfitted, and evaluation loss would rise if trained further.","correct":false,"tempting":"Possible in the next epoch, but it does not explain lower loss with worse accuracy at the current checkpoint."}]},"prev":{"slug":"the-fine-tuning-dataset","title":"Five hundred examples, done properly","url":"https://addaly.com/learn/open-models/the-fine-tuning-dataset"},"next":{"slug":"preference-tuning","title":"Teaching it which answer is better","url":"https://addaly.com/learn/open-models/preference-tuning"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'A real run, on free hardware', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/running-your-first-fine-tune (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"A real run, on free hardware","courseTitle":"The Open Model Ecosystem","module":{"index":6,"slug":"adapting-a-model","title":"Adapting a model to your work"},"answer":null,"keyIdea":"A LoRA run is thirty lines and an hour on free hardware, but the loss curve is only diagnostics: the result is the task metric on a held-out set compared against the base model with your best prompt, at the same settings.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-tools","runs":[{"text":"The tools"}]},{"t":"p","runs":[{"text":"Unsloth","b":true},{"text":": patched kernels giving roughly twice the speed and noticeably less memory than the baseline, with free notebooks for every popular base model. The fastest path from nothing to a trained adapter."}]},{"t":"p","runs":[{"text":"TRL","b":true},{"text":": Hugging Face’s training library. "},{"text":"SFTTrainer","code":true},{"text":" is the standard implementation and what most other tools wrap."}]},{"t":"p","runs":[{"text":"Axolotl","b":true},{"text":": configuration by YAML rather than code. Better for reproducibility and for running many variants."}]},{"t":"p","runs":[{"text":"LLaMA-Factory","b":true},{"text":": a graphical interface over the same ideas, if you would rather not write the script."}]},{"t":"p","runs":[{"text":"All four do the same thing. Pick one and learn its defaults."}]},{"t":"h2","id":"a-complete-run","runs":[{"text":"A complete run"}]},{"t":"code","lang":"python","label":"Python","text":"from datasets import load_dataset\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n\nbase = \"Qwen/Qwen3-8B\"\ntok  = AutoTokenizer.from_pretrained(base)\nmodel = AutoModelForCausalLM.from_pretrained(\n    base, device_map=\"auto\", torch_dtype=\"bfloat16\",\n    quantization_config=BitsAndBytesConfig(load_in_4bit=True,\n        bnb_4bit_quant_type=\"nf4\", bnb_4bit_compute_dtype=\"bfloat16\"),\n)\n\nds = load_dataset(\"json\", data_files={\"train\":\"train.jsonl\",\"test\":\"dev.jsonl\"})\n\ntrainer = SFTTrainer(\n    model=model, tokenizer=tok,\n    train_dataset=ds[\"train\"], eval_dataset=ds[\"test\"],\n    peft_config=LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,\n        target_modules=[\"q_proj\",\"k_proj\",\"v_proj\",\"o_proj\",\n                        \"gate_proj\",\"up_proj\",\"down_proj\"]),\n    args=SFTConfig(\n        output_dir=\"out\", max_seq_length=1024, packing=True,\n        per_device_train_batch_size=1, gradient_accumulation_steps=16,\n        gradient_checkpointing=True, num_train_epochs=2,\n        learning_rate=2e-4, lr_scheduler_type=\"cosine\", warmup_ratio=0.03,\n        logging_steps=10, eval_strategy=\"steps\", eval_steps=50,\n        save_steps=50, save_total_limit=2, bf16=True, seed=42,\n    ),\n)\ntrainer.train()\ntrainer.model.save_pretrained(\"out/adapter\")"},{"t":"p","runs":[{"text":"On 500 examples of about 600 tokens each, two epochs, this takes 30 to 60 minutes on a free notebook GPU."}]},{"t":"h2","id":"the-four-hyperparameters-that-matter","runs":[{"text":"The four hyperparameters that matter"}]},{"t":"p","runs":[{"text":"Learning rate.","b":true},{"text":" For LoRA, "},{"text":"1e-4","code":true},{"text":" to "},{"text":"2e-4","code":true},{"text":". Full fine-tuning wants roughly ten times lower, around "},{"text":"1e-5","code":true},{"text":" to "},{"text":"2e-5","code":true},{"text":", and using a LoRA learning rate for a full tune destroys the model. If the loss goes to zero in fifty steps or turns to "},{"text":"nan","code":true},{"text":", the rate is too high."}]},{"t":"p","runs":[{"text":"Epochs.","b":true},{"text":" One to three on a few hundred examples. Small datasets overfit quickly, and the third epoch is often where the model stops generalising and starts reciting."}]},{"t":"p","runs":[{"text":"Effective batch size.","b":true},{"text":" Batch size times accumulation steps. Sixteen to thirty-two is a reasonable range. Too small makes training noisy; too large wastes the small dataset."}]},{"t":"p","runs":[{"text":"Max sequence length.","b":true},{"text":" Set it to your real data with "},{"text":"packing=True","code":true},{"text":", which fills each sequence with several examples rather than padding. On short examples packing can double throughput."}]},{"t":"p","runs":[{"text":"Everything else can stay at defaults for a first run."}]},{"t":"h2","id":"reading-the-curves-honestly","runs":[{"text":"Reading the curves honestly"}]},{"t":"p","runs":[{"text":"Training loss falling","b":true},{"text":" means the model is fitting your examples. It says nothing about whether it does your task better."}]},{"t":"p","runs":[{"text":"Evaluation loss","b":true},{"text":" is more useful, and the shape to watch for is a fall then a rise: the point where it turns upward is where overfitting begins, and the checkpoint before it is the one you want."}]},{"t":"p","runs":[{"text":"Neither number is your metric.","b":true},{"text":" Loss is average token surprise. Your task is accuracy, or valid JSON rate, or whatever the product needs. A model can have lower loss and be worse for you: for instance because it learned to reproduce a verbose style that raises token-level agreement while making the extraction harder to parse."}]},{"t":"p","runs":[{"text":"Always run the actual task evaluation on the held-out set, at temperature 0, against the base model with your best prompt. Two numbers, same set, same settings. That comparison is the result; the loss curve is diagnostics."}]},{"t":"h2","id":"what-a-failed-run-looks-like","runs":[{"text":"What a failed run looks like"}]},{"t":"ul","items":[[{"text":"Loss barely moves.","b":true},{"text":" Wrong target modules, learning rate far too low, or the loss is being computed on the wrong tokens. Print the trainable parameter count first."}],[{"text":"Loss collapses to near zero in a few steps.","b":true},{"text":" Learning rate too high, or the dataset has near-duplicates being memorised."}],[{"text":"The model outputs the training format for every input, including irrelevant ones.","b":true},{"text":" Overfitting. Fewer epochs, lower rank, more diverse data."}],[{"text":"Output is fluent nonsense.","b":true},{"text":" The chat template used in training does not match the one used at inference. Check both."}]]},{"t":"p","runs":[{"text":"That last one is the most common of all, and the fix is to generate with the exact same template the trainer applied."}]}],"check":{"question":[{"text":"After fine-tuning, evaluation loss is clearly lower than the base model’s but task accuracy on the held-out set is slightly worse. What is the most reasonable interpretation?"}],"options":[{"text":[{"text":"The evaluation set is too small, so the accuracy difference is noise and the loss should be trusted."}],"correct":false,"tempting":[{"text":"Noise is worth checking, but loss and task accuracy measure different things, so agreement between them was never guaranteed."}]},{"text":[{"text":"The learning rate was too high, which lowers loss while damaging the model."}],"correct":false,"tempting":[{"text":"A rate that damages the model raises loss rather than lowering it."}]},{"text":[{"text":"Loss measures token-level agreement with your training style, which the model learned, while accuracy measures the task, so the two can move in opposite directions."}],"correct":true,"tempting":null},{"text":[{"text":"The model has overfitted, and evaluation loss would rise if trained further."}],"correct":false,"tempting":[{"text":"Possible in the next epoch, but it does not explain lower loss with worse accuracy at the current checkpoint."}]}],"order":[0,2,1,3],"why":null},"prev":{"slug":"the-fine-tuning-dataset","title":"Five hundred examples, done properly","minutes":9},"next":{"slug":"preference-tuning","title":"Teaching it which answer is better","minutes":9}}}