Training on a free or nearly free budget

What the free tiers actually give you

Two services matter, and they are not equivalent.

Kaggle Notebooks. Roughly 30 GPU-hours a week, sessions up to 12 hours, and the choice of a single P100, two T4s (16 GB each), or a TPU. The long session limit is the important part: a 12-hour job can finish, where a Colab session usually cannot.

Google Colab, free tier. A T4 with 16 GB, no guarantee of availability, and disconnection after a period of inactivity or at the platform’s discretion. Good for iteration and experimentation; unreliable for a run that needs six uninterrupted hours.

Both give Turing-generation or Pascal-generation cards, which means no bf16. Everything in the precision lesson about fp16 and loss scaling applies to you.

What fits in 16 GB

Measured, not guessed, with gradient checkpointing on and flash attention where supported:

• 7B or 8B, QLoRA, sequence 1,024, batch 1 with accumulation. Fits, with a couple of gigabytes spare. This is the headline configuration and it does work.
• 7B or 8B, QLoRA, sequence 2,048. Marginal. Possible with Unsloth, uncomfortable without.
• 3B, LoRA in fp16 without quantisation. Comfortable.
• 1.5B, LoRA, sequence 2,048, batch 4. Easy.
• 0.5B, full fine-tuning with 8-bit AdamW. Fits, and for a 0.5B model full fine-tuning is usually the better choice anyway.
• A 150M encoder classifier, full fine-tuning. Trains in minutes, and would train on the CPU if you had to.

What does not fit: anything at 70B, full fine-tuning above about 1B, and long-context work. No configuration trick changes those.

Unsloth, and what it is doing

Unsloth rewrites the attention, RoPE, RMSNorm and cross-entropy kernels for LoRA and QLoRA training in Triton, and manually derives the backward passes rather than relying on autograd. The result on a single card is roughly two times faster training and substantially lower memory, with the same mathematics. The single-GPU version is open source and free.

On a free T4 this is not a nicety. It is often the difference between a configuration that fits and one that does not, and between a run that finishes in your session and one that does not.

from unsloth import FastLanguageModel
model, tok = FastLanguageModel.from_pretrained(
    "unsloth/llama-3.1-8b-bnb-4bit", max_seq_length=1024, load_in_4bit=True)
model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32,
            target_modules=["q_proj","k_proj","v_proj","o_proj",
                            "gate_proj","up_proj","down_proj"],
            use_gradient_checkpointing="unsloth")

Llama-Factory and Axolotl are the other free, mature options; both are configuration-file driven, both wrap the same underlying libraries, and both are worth preferring over a hand-written loop for a first project.

Surviving a session that will be killed

Assume the session dies. Two habits make that a nuisance rather than a disaster:

TrainingArguments(save_steps=200, save_total_limit=3,
                  output_dir="/content/drive/MyDrive/run-01")

Save to mounted Drive or push to the Hugging Face Hub, not to the ephemeral local disk. And set resume_from_checkpoint=True so restarting the notebook continues rather than restarting. A LoRA checkpoint is a few hundred megabytes, so keeping three of them is not a storage problem.

When you do pay, pay properly

Rented compute is cheaper than most people expect, and the community marketplaces are cheaper than the hyperscalers by a wide margin.

• An RTX 4090 (24 GB) on a community marketplace: roughly $0.30 to $0.60 an hour.
• An A100 40 GB: roughly $1.00 to $1.80 an hour.
• Interruptible or spot instances are typically half of those, and your checkpointing habit already makes interruption survivable.

A concrete job: 3,000 examples, 8B model, QLoRA, three epochs, sequence 1,024. That is around 560 optimiser steps at an effective batch of 16, and on a 4090 it finishes in one to two hours. Under a pound. The GPU has never been the expensive part of this; the dataset and the evaluation are, and they are paid in attention rather than in currency.

The free path, end to end

Build the dataset in a text editor. Train the adapter on Kaggle. Store checkpoints on the Hugging Face Hub, free for public repositories. Merge and convert to GGUF, then run the result on your own laptop with llama.cpp or Ollama: a 7B model in 4-bit needs about 5 GB of RAM and runs on a CPU at a few tokens per second, which is slow but is a real, working, private model on hardware you already own.

Nothing in that chain costs money. It is slower than the rented version at every step, and it produces the same artefact.