A real run, on free hardware

The tools

Unsloth: patched kernels giving roughly twice the speed and noticeably less memory than the baseline, with free notebooks for every popular base model. The fastest path from nothing to a trained adapter.

TRL: Hugging Face’s training library. SFTTrainer is the standard implementation and what most other tools wrap.

Axolotl: configuration by YAML rather than code. Better for reproducibility and for running many variants.

LLaMA-Factory: a graphical interface over the same ideas, if you would rather not write the script.

All four do the same thing. Pick one and learn its defaults.

A complete run

from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
from peft import LoraConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base = "Qwen/Qwen3-8B"
tok  = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base, device_map="auto", torch_dtype="bfloat16",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True,
        bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype="bfloat16"),
)

ds = load_dataset("json", data_files={"train":"train.jsonl","test":"dev.jsonl"})

trainer = SFTTrainer(
    model=model, tokenizer=tok,
    train_dataset=ds["train"], eval_dataset=ds["test"],
    peft_config=LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
        target_modules=["q_proj","k_proj","v_proj","o_proj",
                        "gate_proj","up_proj","down_proj"]),
    args=SFTConfig(
        output_dir="out", max_seq_length=1024, packing=True,
        per_device_train_batch_size=1, gradient_accumulation_steps=16,
        gradient_checkpointing=True, num_train_epochs=2,
        learning_rate=2e-4, lr_scheduler_type="cosine", warmup_ratio=0.03,
        logging_steps=10, eval_strategy="steps", eval_steps=50,
        save_steps=50, save_total_limit=2, bf16=True, seed=42,
    ),
)
trainer.train()
trainer.model.save_pretrained("out/adapter")

On 500 examples of about 600 tokens each, two epochs, this takes 30 to 60 minutes on a free notebook GPU.

The four hyperparameters that matter

Learning rate. For LoRA, 1e-4 to 2e-4. Full fine-tuning wants roughly ten times lower, around 1e-5 to 2e-5, and using a LoRA learning rate for a full tune destroys the model. If the loss goes to zero in fifty steps or turns to nan, the rate is too high.

Epochs. One to three on a few hundred examples. Small datasets overfit quickly, and the third epoch is often where the model stops generalising and starts reciting.

Effective batch size. Batch size times accumulation steps. Sixteen to thirty-two is a reasonable range. Too small makes training noisy; too large wastes the small dataset.

Max sequence length. Set it to your real data with packing=True, which fills each sequence with several examples rather than padding. On short examples packing can double throughput.

Everything else can stay at defaults for a first run.

Reading the curves honestly

Training loss falling means the model is fitting your examples. It says nothing about whether it does your task better.

Evaluation loss is more useful, and the shape to watch for is a fall then a rise: the point where it turns upward is where overfitting begins, and the checkpoint before it is the one you want.

Neither number is your metric. Loss is average token surprise. Your task is accuracy, or valid JSON rate, or whatever the product needs. A model can have lower loss and be worse for you: for instance because it learned to reproduce a verbose style that raises token-level agreement while making the extraction harder to parse.

Always run the actual task evaluation on the held-out set, at temperature 0, against the base model with your best prompt. Two numbers, same set, same settings. That comparison is the result; the loss curve is diagnostics.

What a failed run looks like

• Loss barely moves. Wrong target modules, learning rate far too low, or the loss is being computed on the wrong tokens. Print the trainable parameter count first.
• Loss collapses to near zero in a few steps. Learning rate too high, or the dataset has near-duplicates being memorised.
• The model outputs the training format for every input, including irrelevant ones. Overfitting. Fewer epochs, lower rank, more diverse data.
• Output is fluent nonsense. The chat template used in training does not match the one used at inference. Check both.

That last one is the most common of all, and the fix is to generate with the exact same template the trainer applied.