{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"training-on-a-small-budget","title":"Training on a free or nearly free budget","url":"https://addaly.com/learn/fine-tuning/training-on-a-small-budget","position":27,"of":84,"minutes":10,"answer":"A free Kaggle T4 with 12-hour sessions trains an 8B QLoRA at sequence 1,024, Unsloth's rewritten kernels roughly double that throughput.","keyIdea":"A free Kaggle T4 with 12-hour sessions trains an 8B QLoRA at sequence 1,024, Unsloth's rewritten kernels roughly double that throughput, and the whole chain from dataset to a GGUF model running on your own laptop costs nothing, with checkpointing to Drive or the Hub being what makes a killable session survivable."},"body":{"format":"markdown","text":"## What the free tiers actually give you\n\nTwo services matter, and they are not equivalent.\n\n**Kaggle Notebooks.** Roughly 30 GPU-hours a week, sessions up to 12 hours, and the choice of a single P100, two T4s (16 GB each), or a TPU. The long session limit is the important part: a 12-hour job can finish, where a Colab session usually cannot.\n\n**Google Colab, free tier.** A T4 with 16 GB, no guarantee of availability, and disconnection after a period of inactivity or at the platform's discretion. Good for iteration and experimentation; unreliable for a run that needs six uninterrupted hours.\n\nBoth give Turing-generation or Pascal-generation cards, which means **no bf16**. Everything in the precision lesson about fp16 and loss scaling applies to you.\n\n## What fits in 16 GB\n\nMeasured, not guessed, with gradient checkpointing on and flash attention where supported:\n\n- **7B or 8B, QLoRA, sequence 1,024, batch 1 with accumulation.** Fits, with a couple of gigabytes spare. This is the headline configuration and it does work.\n- **7B or 8B, QLoRA, sequence 2,048.** Marginal. Possible with Unsloth, uncomfortable without.\n- **3B, LoRA in fp16 without quantisation.** Comfortable.\n- **1.5B, LoRA, sequence 2,048, batch 4.** Easy.\n- **0.5B, full fine-tuning with 8-bit AdamW.** Fits, and for a 0.5B model full fine-tuning is usually the better choice anyway.\n- **A 150M encoder classifier, full fine-tuning.** Trains in minutes, and would train on the CPU if you had to.\n\nWhat does not fit: anything at 70B, full fine-tuning above about 1B, and long-context work. No configuration trick changes those.\n\n## Unsloth, and what it is doing\n\nUnsloth rewrites the attention, RoPE, RMSNorm and cross-entropy kernels for LoRA and QLoRA training in Triton, and manually derives the backward passes rather than relying on autograd. The result on a single card is roughly two times faster training and substantially lower memory, with the same mathematics. The single-GPU version is open source and free.\n\nOn a free T4 this is not a nicety. It is often the difference between a configuration that fits and one that does not, and between a run that finishes in your session and one that does not.\n\n```python\nfrom unsloth import FastLanguageModel\nmodel, tok = FastLanguageModel.from_pretrained(\n    \"unsloth/llama-3.1-8b-bnb-4bit\", max_seq_length=1024, load_in_4bit=True)\nmodel = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32,\n            target_modules=[\"q_proj\",\"k_proj\",\"v_proj\",\"o_proj\",\n                            \"gate_proj\",\"up_proj\",\"down_proj\"],\n            use_gradient_checkpointing=\"unsloth\")\n```\n\nLlama-Factory and Axolotl are the other free, mature options; both are configuration-file driven, both wrap the same underlying libraries, and both are worth preferring over a hand-written loop for a first project.\n\n## Surviving a session that will be killed\n\nAssume the session dies. Two habits make that a nuisance rather than a disaster:\n\n```python\nTrainingArguments(save_steps=200, save_total_limit=3,\n                  output_dir=\"/content/drive/MyDrive/run-01\")\n```\n\nSave to mounted Drive or push to the Hugging Face Hub, not to the ephemeral local disk. And set `resume_from_checkpoint=True` so restarting the notebook continues rather than restarting. A LoRA checkpoint is a few hundred megabytes, so keeping three of them is not a storage problem.\n\n## When you do pay, pay properly\n\nRented compute is cheaper than most people expect, and the community marketplaces are cheaper than the hyperscalers by a wide margin.\n\n- An RTX 4090 (24 GB) on a community marketplace: roughly $0.30 to $0.60 an hour.\n- An A100 40 GB: roughly $1.00 to $1.80 an hour.\n- Interruptible or spot instances are typically half of those, and your checkpointing habit already makes interruption survivable.\n\nA concrete job: 3,000 examples, 8B model, QLoRA, three epochs, sequence 1,024. That is around 560 optimiser steps at an effective batch of 16, and on a 4090 it finishes in one to two hours. **Under a pound.** The GPU has never been the expensive part of this; the dataset and the evaluation are, and they are paid in attention rather than in currency.\n\n## The free path, end to end\n\nBuild the dataset in a text editor. Train the adapter on Kaggle. Store checkpoints on the Hugging Face Hub, free for public repositories. Merge and convert to GGUF, then run the result on your own laptop with `llama.cpp` or Ollama: a 7B model in 4-bit needs about 5 GB of RAM and runs on a CPU at a few tokens per second, which is slow but is a real, working, private model on hardware you already own.\n\nNothing in that chain costs money. It is slower than the rented version at every step, and it produces the same artefact."},"check":{"question":"Someone runs an 8B QLoRA on a free Colab T4 and sets bnb_4bit_compute_dtype=torch.bfloat16 after copying a config from a blog. What happens?","options":[{"text":"Nothing changes; bf16 and fp16 are interchangeable at 4-bit quantisation because the weights are stored in NF4 either way.","correct":false,"tempting":"Assumes the storage format determines the compute path, when the compute dtype governs the dequantised matrix multiplications."},{"text":"Memory use doubles, because bf16 requires an fp32 master copy that fp16 does not.","correct":false,"tempting":"Both mixed-precision paths keep fp32 master weights for trainable parameters; the difference between them is not a master-copy requirement."},{"text":"Training proceeds correctly but the resulting adapter can only be merged into a bf16 base, making it unusable on the same T4 afterwards.","correct":false,"tempting":"Invents a compatibility constraint on the artefact; adapter weights are dtype-convertible and carry no such restriction."},{"text":"The T4 is Turing and has no bf16, so the setting errors or is emulated, and fp16 is what the hardware wants.","correct":true}]},"prev":{"slug":"full-fine-tuning-when-right","title":"Full fine-tuning, when it is genuinely right","url":"https://addaly.com/learn/fine-tuning/full-fine-tuning-when-right"},"next":{"slug":"the-dataset","title":"The Dataset Decides Everything","url":"https://addaly.com/learn/fine-tuning/the-dataset"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Training on a free or nearly free budget', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/training-on-a-small-budget (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Training on a free or nearly free budget","courseTitle":"Fine-Tuning, and When Not To","module":{"index":3,"slug":"the-peft-family","title":"Full, LoRA and the parameter-efficient family"},"answer":null,"keyIdea":"A free Kaggle T4 with 12-hour sessions trains an 8B QLoRA at sequence 1,024, Unsloth’s rewritten kernels roughly double that throughput, and the whole chain from dataset to a GGUF model running on your own laptop costs nothing, with checkpointing to Drive or the Hub being what makes a killable session survivable.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-the-free-tiers-actually-give-you","runs":[{"text":"What the free tiers actually give you"}]},{"t":"p","runs":[{"text":"Two services matter, and they are not equivalent."}]},{"t":"p","runs":[{"text":"Kaggle Notebooks.","b":true},{"text":" Roughly 30 GPU-hours a week, sessions up to 12 hours, and the choice of a single P100, two T4s (16 GB each), or a TPU. The long session limit is the important part: a 12-hour job can finish, where a Colab session usually cannot."}]},{"t":"p","runs":[{"text":"Google Colab, free tier.","b":true},{"text":" A T4 with 16 GB, no guarantee of availability, and disconnection after a period of inactivity or at the platform’s discretion. Good for iteration and experimentation; unreliable for a run that needs six uninterrupted hours."}]},{"t":"p","runs":[{"text":"Both give Turing-generation or Pascal-generation cards, which means "},{"text":"no bf16","b":true},{"text":". Everything in the precision lesson about fp16 and loss scaling applies to you."}]},{"t":"h2","id":"what-fits-in-16-gb","runs":[{"text":"What fits in 16 GB"}]},{"t":"p","runs":[{"text":"Measured, not guessed, with gradient checkpointing on and flash attention where supported:"}]},{"t":"ul","items":[[{"text":"7B or 8B, QLoRA, sequence 1,024, batch 1 with accumulation.","b":true},{"text":" Fits, with a couple of gigabytes spare. This is the headline configuration and it does work."}],[{"text":"7B or 8B, QLoRA, sequence 2,048.","b":true},{"text":" Marginal. Possible with Unsloth, uncomfortable without."}],[{"text":"3B, LoRA in fp16 without quantisation.","b":true},{"text":" Comfortable."}],[{"text":"1.5B, LoRA, sequence 2,048, batch 4.","b":true},{"text":" Easy."}],[{"text":"0.5B, full fine-tuning with 8-bit AdamW.","b":true},{"text":" Fits, and for a 0.5B model full fine-tuning is usually the better choice anyway."}],[{"text":"A 150M encoder classifier, full fine-tuning.","b":true},{"text":" Trains in minutes, and would train on the CPU if you had to."}]]},{"t":"p","runs":[{"text":"What does not fit: anything at 70B, full fine-tuning above about 1B, and long-context work. No configuration trick changes those."}]},{"t":"h2","id":"unsloth-and-what-it-is-doing","runs":[{"text":"Unsloth, and what it is doing"}]},{"t":"p","runs":[{"text":"Unsloth rewrites the attention, RoPE, RMSNorm and cross-entropy kernels for LoRA and QLoRA training in Triton, and manually derives the backward passes rather than relying on autograd. The result on a single card is roughly two times faster training and substantially lower memory, with the same mathematics. The single-GPU version is open source and free."}]},{"t":"p","runs":[{"text":"On a free T4 this is not a nicety. It is often the difference between a configuration that fits and one that does not, and between a run that finishes in your session and one that does not."}]},{"t":"code","lang":"python","label":"Python","text":"from unsloth import FastLanguageModel\nmodel, tok = FastLanguageModel.from_pretrained(\n    \"unsloth/llama-3.1-8b-bnb-4bit\", max_seq_length=1024, load_in_4bit=True)\nmodel = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32,\n            target_modules=[\"q_proj\",\"k_proj\",\"v_proj\",\"o_proj\",\n                            \"gate_proj\",\"up_proj\",\"down_proj\"],\n            use_gradient_checkpointing=\"unsloth\")"},{"t":"p","runs":[{"text":"Llama-Factory and Axolotl are the other free, mature options; both are configuration-file driven, both wrap the same underlying libraries, and both are worth preferring over a hand-written loop for a first project."}]},{"t":"h2","id":"surviving-a-session-that-will-be-killed","runs":[{"text":"Surviving a session that will be killed"}]},{"t":"p","runs":[{"text":"Assume the session dies. Two habits make that a nuisance rather than a disaster:"}]},{"t":"code","lang":"python","label":"Python","text":"TrainingArguments(save_steps=200, save_total_limit=3,\n                  output_dir=\"/content/drive/MyDrive/run-01\")"},{"t":"p","runs":[{"text":"Save to mounted Drive or push to the Hugging Face Hub, not to the ephemeral local disk. And set "},{"text":"resume_from_checkpoint=True","code":true},{"text":" so restarting the notebook continues rather than restarting. A LoRA checkpoint is a few hundred megabytes, so keeping three of them is not a storage problem."}]},{"t":"h2","id":"when-you-do-pay-pay-properly","runs":[{"text":"When you do pay, pay properly"}]},{"t":"p","runs":[{"text":"Rented compute is cheaper than most people expect, and the community marketplaces are cheaper than the hyperscalers by a wide margin."}]},{"t":"ul","items":[[{"text":"An RTX 4090 (24 GB) on a community marketplace: roughly $0.30 to $0.60 an hour."}],[{"text":"An A100 40 GB: roughly $1.00 to $1.80 an hour."}],[{"text":"Interruptible or spot instances are typically half of those, and your checkpointing habit already makes interruption survivable."}]]},{"t":"p","runs":[{"text":"A concrete job: 3,000 examples, 8B model, QLoRA, three epochs, sequence 1,024. That is around 560 optimiser steps at an effective batch of 16, and on a 4090 it finishes in one to two hours. "},{"text":"Under a pound.","b":true},{"text":" The GPU has never been the expensive part of this; the dataset and the evaluation are, and they are paid in attention rather than in currency."}]},{"t":"h2","id":"the-free-path-end-to-end","runs":[{"text":"The free path, end to end"}],"role":"free"},{"t":"p","runs":[{"text":"Build the dataset in a text editor. Train the adapter on Kaggle. Store checkpoints on the Hugging Face Hub, free for public repositories. Merge and convert to GGUF, then run the result on your own laptop with "},{"text":"llama.cpp","code":true},{"text":" or Ollama: a 7B model in 4-bit needs about 5 GB of RAM and runs on a CPU at a few tokens per second, which is slow but is a real, working, private model on hardware you already own."}]},{"t":"p","runs":[{"text":"Nothing in that chain costs money. It is slower than the rented version at every step, and it produces the same artefact."}]}],"check":{"question":[{"text":"Someone runs an 8B QLoRA on a free Colab T4 and sets bnb_4bit_compute_dtype=torch.bfloat16 after copying a config from a blog. What happens?"}],"options":[{"text":[{"text":"Nothing changes; bf16 and fp16 are interchangeable at 4-bit quantisation because the weights are stored in NF4 either way."}],"correct":false,"tempting":[{"text":"Assumes the storage format determines the compute path, when the compute dtype governs the dequantised matrix multiplications."}]},{"text":[{"text":"Memory use doubles, because bf16 requires an fp32 master copy that fp16 does not."}],"correct":false,"tempting":[{"text":"Both mixed-precision paths keep fp32 master weights for trainable parameters; the difference between them is not a master-copy requirement."}]},{"text":[{"text":"Training proceeds correctly but the resulting adapter can only be merged into a bf16 base, making it unusable on the same T4 afterwards."}],"correct":false,"tempting":[{"text":"Invents a compatibility constraint on the artefact; adapter weights are dtype-convertible and carry no such restriction."}]},{"text":[{"text":"The T4 is Turing and has no bf16, so the setting errors or is emulated, and fp16 is what the hardware wants."}],"correct":true,"tempting":null}],"order":[1,2,0,3],"why":null},"prev":{"slug":"full-fine-tuning-when-right","title":"Full fine-tuning, when it is genuinely right","minutes":9},"next":{"slug":"the-dataset","title":"The Dataset Decides Everything","minutes":11}}}