{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"running-the-training-run","title":"Running the training run","url":"https://addaly.com/learn/running-models-yourself/running-the-training-run","position":78,"of":82,"minutes":9,"answer":"Learning rate around 1e-4 to 2e-4, two epochs and an effective batch of sixteen to thirty-two covers most LoRA runs.","keyIdea":"Learning rate around 1e-4 to 2e-4, two epochs and an effective batch of sixteen to thirty-two covers most LoRA runs, and printing the decoded tokens that actually contribute to the loss before launching catches the data-formatting bugs that cause most first-run failures."},"body":{"format":"markdown","text":"## The tools, all free\n\nFour options cover essentially everything, and none of them costs money.\n\n**Unsloth** is the fastest single-GPU path and the lowest-friction. It patches the attention and MLP kernels for the common architectures and typically trains roughly twice as fast with noticeably less memory than a stock setup. Apache-2.0 licensed. Its limitation is coverage: it supports a list of architectures, and a model outside that list needs something else.\n\n**TRL with PEFT** is the Hugging Face path. Most flexible, best documented, works with any architecture that loads, slower than Unsloth. This is what to learn if you want to understand what is happening.\n\n**Axolotl** is configuration-driven: you write a YAML file instead of a script. Excellent for reproducibility and for running many variants, and it hides mechanics you may want to see the first time.\n\n**llama.cpp** includes a fine-tuning path that runs on CPU and on modest hardware. It is slow and limited, and for someone with no GPU at all and no access to a free notebook tier, it is a real option rather than a theoretical one.\n\n## A run that works\n\n```python\nfrom trl import SFTTrainer, SFTConfig\n\nargs = SFTConfig(\n    output_dir=\"out\",\n    num_train_epochs=2,\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=16,      # effective batch size 16\n    learning_rate=2e-4,                  # LoRA rates are ~10x full fine-tuning\n    lr_scheduler_type=\"cosine\",\n    warmup_ratio=0.03,\n    max_seq_length=2048,\n    logging_steps=10,\n    eval_strategy=\"steps\", eval_steps=50,\n    save_steps=50, save_total_limit=3,\n    optim=\"paged_adamw_8bit\",\n    bf16=True,\n    seed=3407,\n)\ntrainer = SFTTrainer(model=model, args=args, peft_config=cfg,\n                     train_dataset=train_ds, eval_dataset=eval_ds)\ntrainer.train()\n```\n\n## The five settings that matter, in order\n\n**Learning rate.** LoRA tolerates and needs a much higher rate than full fine-tuning, because only a small low-rank update is being learned. Between `1e-4` and `2e-4` is the working range for most adapters. Below `5e-5` the adapter often barely moves; above `5e-4` runs become unstable and the loss curve shows it.\n\n**Epochs.** Two is a good default; three is often the most a few hundred examples will take before memorising them. If you find yourself wanting five, the honest reading is that the dataset is too small rather than that the model needs longer.\n\n**Effective batch size.** Batch size times accumulation steps. Sixteen to thirty-two is a reasonable range. Very small effective batches make the loss curve noisy and the run harder to read.\n\n**Warmup.** A few per cent of total steps. Cheap insurance against an early instability.\n\n**Sequence length.** Set by your data and your memory, as the previous lesson described. Longer than your data needs is wasted memory.\n\nNotice what is not on this list. Rank, dropout, and the scheduler shape all matter less than any of the above, and people spend disproportionate time on them.\n\n## Print three examples before you launch\n\nThe highest-value thirty seconds in this whole process:\n\n```python\nfor i in range(3):\n    ids = train_ds[i][\"input_ids\"]\n    labels = train_ds[i][\"labels\"]\n    print(tok.decode(ids))\n    print(\"TRAINED ON:\", tok.decode([t for t in labels if t != -100]))\n    print(\"---\")\n```\n\nThat second line is the one that catches real bugs. It shows exactly which tokens contribute to the loss. If it prints the user's message, your masking is wrong. If it prints nothing, your masking is inverted and the run will do nothing for two hours. If it prints the assistant's reply, you are ready.\n\nMost first runs fail on data formatting, not on hyperparameters, and this check finds nearly all of it.\n\n## What the run should look like\n\nOn a consumer card, a 7B QLoRA over about 1,000 examples at 2,048 tokens for two epochs is typically well under an hour. If your estimate is wildly different from that shape (eight hours, or ninety seconds) something is wrong with the data size, the sequence length, or whether the GPU is being used at all. Check `nvidia-smi` during the first minute: utilisation should be high and steady.\n\nCheckpoint every 50 steps with `save_total_limit` so a crash costs minutes rather than the run. Set the seed and record it, along with the base model revision, the dataset hash and the full configuration. A fine-tune you cannot reproduce is a result you cannot defend, and \"which version of the data was that\" is a question you will be asked.\n\n> The run is the easy part. Everything that decides whether it worked happened before it started, or happens in the evaluation afterwards."},"check":{"question":"A first LoRA run completes without error, and the resulting adapter produces output indistinguishable from the base model. The loss was reported as very close to zero from the first step. What should be checked first?","options":[{"text":"Whether loss masking is inverted, so that no tokens contribute to the loss and the run trained on nothing.","correct":true},{"text":"The learning rate, which is probably far too low for the adapter to move from its zero initialisation.","correct":false,"tempting":"A low rate produces a slowly falling loss, not a near-zero loss from step one, and the reported curve rules it out."},{"text":"The adapter rank, which may be too low to express any useful update on this task.","correct":false,"tempting":"Even rank 4 changes behaviour detectably; a rank that small underfits rather than producing an adapter identical to the base."},{"text":"Whether the adapter was saved and reloaded correctly, since a mismatched path would silently serve the base weights.","correct":false,"tempting":"Worth verifying in general, but it would not explain a near-zero training loss from the very first step."}]},"prev":{"slug":"building-the-dataset","title":"The dataset is the project","url":"https://addaly.com/learn/running-models-yourself/building-the-dataset"},"next":{"slug":"reading-the-loss-curve","title":"Generalisation, memorisation and forgetting","url":"https://addaly.com/learn/running-models-yourself/reading-the-loss-curve"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Running the training run', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/running-the-training-run (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Running the training run","courseTitle":"Running Models Yourself","module":{"index":9,"slug":"making-it-yours","title":"Making it yours: adapters and training"},"answer":null,"keyIdea":"Learning rate around 1e-4 to 2e-4, two epochs and an effective batch of sixteen to thirty-two covers most LoRA runs, and printing the decoded tokens that actually contribute to the loss before launching catches the data-formatting bugs that cause most first-run failures.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-tools-all-free","runs":[{"text":"The tools, all free"}]},{"t":"p","runs":[{"text":"Four options cover essentially everything, and none of them costs money."}]},{"t":"p","runs":[{"text":"Unsloth","b":true},{"text":" is the fastest single-GPU path and the lowest-friction. It patches the attention and MLP kernels for the common architectures and typically trains roughly twice as fast with noticeably less memory than a stock setup. Apache-2.0 licensed. Its limitation is coverage: it supports a list of architectures, and a model outside that list needs something else."}]},{"t":"p","runs":[{"text":"TRL with PEFT","b":true},{"text":" is the Hugging Face path. Most flexible, best documented, works with any architecture that loads, slower than Unsloth. This is what to learn if you want to understand what is happening."}]},{"t":"p","runs":[{"text":"Axolotl","b":true},{"text":" is configuration-driven: you write a YAML file instead of a script. Excellent for reproducibility and for running many variants, and it hides mechanics you may want to see the first time."}]},{"t":"p","runs":[{"text":"llama.cpp","b":true},{"text":" includes a fine-tuning path that runs on CPU and on modest hardware. It is slow and limited, and for someone with no GPU at all and no access to a free notebook tier, it is a real option rather than a theoretical one."}]},{"t":"h2","id":"a-run-that-works","runs":[{"text":"A run that works"}]},{"t":"code","lang":"python","label":"Python","text":"from trl import SFTTrainer, SFTConfig\n\nargs = SFTConfig(\n    output_dir=\"out\",\n    num_train_epochs=2,\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=16,      # effective batch size 16\n    learning_rate=2e-4,                  # LoRA rates are ~10x full fine-tuning\n    lr_scheduler_type=\"cosine\",\n    warmup_ratio=0.03,\n    max_seq_length=2048,\n    logging_steps=10,\n    eval_strategy=\"steps\", eval_steps=50,\n    save_steps=50, save_total_limit=3,\n    optim=\"paged_adamw_8bit\",\n    bf16=True,\n    seed=3407,\n)\ntrainer = SFTTrainer(model=model, args=args, peft_config=cfg,\n                     train_dataset=train_ds, eval_dataset=eval_ds)\ntrainer.train()"},{"t":"h2","id":"the-five-settings-that-matter-in-order","runs":[{"text":"The five settings that matter, in order"}]},{"t":"p","runs":[{"text":"Learning rate.","b":true},{"text":" LoRA tolerates and needs a much higher rate than full fine-tuning, because only a small low-rank update is being learned. Between "},{"text":"1e-4","code":true},{"text":" and "},{"text":"2e-4","code":true},{"text":" is the working range for most adapters. Below "},{"text":"5e-5","code":true},{"text":" the adapter often barely moves; above "},{"text":"5e-4","code":true},{"text":" runs become unstable and the loss curve shows it."}]},{"t":"p","runs":[{"text":"Epochs.","b":true},{"text":" Two is a good default; three is often the most a few hundred examples will take before memorising them. If you find yourself wanting five, the honest reading is that the dataset is too small rather than that the model needs longer."}]},{"t":"p","runs":[{"text":"Effective batch size.","b":true},{"text":" Batch size times accumulation steps. Sixteen to thirty-two is a reasonable range. Very small effective batches make the loss curve noisy and the run harder to read."}]},{"t":"p","runs":[{"text":"Warmup.","b":true},{"text":" A few per cent of total steps. Cheap insurance against an early instability."}]},{"t":"p","runs":[{"text":"Sequence length.","b":true},{"text":" Set by your data and your memory, as the previous lesson described. Longer than your data needs is wasted memory."}]},{"t":"p","runs":[{"text":"Notice what is not on this list. Rank, dropout, and the scheduler shape all matter less than any of the above, and people spend disproportionate time on them."}]},{"t":"h2","id":"print-three-examples-before-you-launch","runs":[{"text":"Print three examples before you launch"}]},{"t":"p","runs":[{"text":"The highest-value thirty seconds in this whole process:"}]},{"t":"code","lang":"python","label":"Python","text":"for i in range(3):\n    ids = train_ds[i][\"input_ids\"]\n    labels = train_ds[i][\"labels\"]\n    print(tok.decode(ids))\n    print(\"TRAINED ON:\", tok.decode([t for t in labels if t != -100]))\n    print(\"---\")"},{"t":"p","runs":[{"text":"That second line is the one that catches real bugs. It shows exactly which tokens contribute to the loss. If it prints the user’s message, your masking is wrong. If it prints nothing, your masking is inverted and the run will do nothing for two hours. If it prints the assistant’s reply, you are ready."}]},{"t":"p","runs":[{"text":"Most first runs fail on data formatting, not on hyperparameters, and this check finds nearly all of it."}]},{"t":"h2","id":"what-the-run-should-look-like","runs":[{"text":"What the run should look like"}]},{"t":"p","runs":[{"text":"On a consumer card, a 7B QLoRA over about 1,000 examples at 2,048 tokens for two epochs is typically well under an hour. If your estimate is wildly different from that shape (eight hours, or ninety seconds) something is wrong with the data size, the sequence length, or whether the GPU is being used at all. Check "},{"text":"nvidia-smi","code":true},{"text":" during the first minute: utilisation should be high and steady."}]},{"t":"p","runs":[{"text":"Checkpoint every 50 steps with "},{"text":"save_total_limit","code":true},{"text":" so a crash costs minutes rather than the run. Set the seed and record it, along with the base model revision, the dataset hash and the full configuration. A fine-tune you cannot reproduce is a result you cannot defend, and “which version of the data was that” is a question you will be asked."}]},{"t":"quote","runs":[{"text":"The run is the easy part. Everything that decides whether it worked happened before it started, or happens in the evaluation afterwards."}]}],"check":{"question":[{"text":"A first LoRA run completes without error, and the resulting adapter produces output indistinguishable from the base model. The loss was reported as very close to zero from the first step. What should be checked first?"}],"options":[{"text":[{"text":"Whether loss masking is inverted, so that no tokens contribute to the loss and the run trained on nothing."}],"correct":true,"tempting":null},{"text":[{"text":"The learning rate, which is probably far too low for the adapter to move from its zero initialisation."}],"correct":false,"tempting":[{"text":"A low rate produces a slowly falling loss, not a near-zero loss from step one, and the reported curve rules it out."}]},{"text":[{"text":"The adapter rank, which may be too low to express any useful update on this task."}],"correct":false,"tempting":[{"text":"Even rank 4 changes behaviour detectably; a rank that small underfits rather than producing an adapter identical to the base."}]},{"text":[{"text":"Whether the adapter was saved and reloaded correctly, since a mismatched path would silently serve the base weights."}],"correct":false,"tempting":[{"text":"Worth verifying in general, but it would not explain a near-zero training loss from the very first step."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"building-the-dataset","title":"The dataset is the project","minutes":9},"next":{"slug":"reading-the-loss-curve","title":"Generalisation, memorisation and forgetting","minutes":9}}}