How a number inside the model gets changed

The loop

Everything in training is this, several hundred thousand times:

1. Take a batch of text.
2. Run it forward; compute the loss.
3. Compute the gradient: for every one of the 8 billion parameters, how much would the loss change if this parameter changed slightly.
4. Move every parameter a small step in the direction that reduces the loss.
5. Repeat.

Step 3 is backpropagation, and it is not magic. It is the chain rule from calculus applied systematically backwards through the network, reusing intermediate results so that computing all 8 billion partial derivatives costs about twice a forward pass rather than 8 billion times.

That efficiency is the reason deep learning is possible at all. Without it, the derivative for each parameter would need its own forward pass and training would be unthinkable.

The learning rate, and why it is the one you tune

Step 4 needs a size. Too large and the update overshoots, the loss oscillates or explodes. Too small and training takes forever and can settle into a poor region.

In practice the learning rate follows a schedule: a short warmup from near zero over the first few thousand steps, then a peak, then a decay (usually cosine) down toward zero by the end. The warmup exists because early gradients are large and unreliable; the decay lets the model settle.

Typical peak values are around 10⁻⁴ for pretraining a mid-size model and 10⁻⁵ or lower for fine-tuning. When a fine-tune produces nonsense, the learning rate being too high is the first suspect, every time.

Adam, and the memory it costs

Plain gradient descent uses the gradient directly. Almost nothing does that any more. Adam and its variants keep two extra running averages per parameter (one of the gradient, one of its square) which lets each parameter get an effectively different step size.

The cost is memory, and this is where training budgets come from. Per parameter, in a standard mixed-precision setup:

• 2 bytes for the weight in bf16
• 4 bytes for a master copy in fp32
• 4 bytes for the first moment
• 4 bytes for the second moment
• 2 bytes for the gradient

About 16 bytes per parameter, against 2 bytes for inference. An 8B model needs roughly 128 GB of optimiser and weight state before a single activation is stored, which is why fine-tuning a model that runs happily on one GPU may need eight.

Batches, and why they are huge

Gradients from a single example are noisy. Averaging over many gives a more reliable direction, and large batches keep the hardware busy. Pretraining batches of one to four million tokens are normal.

You rarely have the memory to hold that at once, so gradient accumulation is used: run several small batches, accumulate the gradients, then update once. Same result, less memory, more time.

Activations, the hidden cost

The backward pass needs the intermediate values from the forward pass, so they must be kept. For long sequences and large batches this can exceed the weights themselves. Gradient checkpointing trades compute for memory: discard most intermediates and recompute them during the backward pass, roughly 30 per cent more compute for a large memory saving. Almost every fine-tuning script offers a flag for it, and turning it on is usually the difference between fitting and not.

Why this matters even if you never train

Three practical consequences.

Inference and training costs differ by an order of magnitude. A model you can serve on one GPU may need a cluster to train. When someone proposes fine-tuning, the memory arithmetic above is the first check, and it is the reason the parameter-efficient methods in a later lesson exist.

Weights do not change during use. The loop above runs during training and not during a conversation. Nothing you say updates anything.

A model cannot learn from its mistakes in production without a deliberate retraining cycle: collect examples, curate them, run the loop, evaluate, deploy. That is a project, not a feature, and understanding why is the difference between a realistic roadmap and a wishful one.