{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"one-training-step","title":"One training step, taken apart","url":"https://addaly.com/learn/fine-tuning/one-training-step","position":10,"of":84,"minutes":10,"answer":"A training step is forward, cross-entropy against the next token, backward to gradients, optimiser update, and the memory, the masking and every hyperparameter you will set are consequences of those four lines.","keyIdea":"A training step is forward, cross-entropy against the next token, backward to gradients, optimiser update, and the memory, the masking and every hyperparameter you will set are consequences of those four lines."},"body":{"format":"markdown","text":"## Four things happen, in order\n\nEverything in this course is a variation on one step, repeated a few thousand times. The step has four parts: a forward pass, a loss, a backward pass, and an update. If you can narrate these with real shapes, every hyperparameter later stops being folklore.\n\nTake a concrete batch: 4 sequences of 1,024 tokens each, on an 8B model with a vocabulary of 128,256.\n\n## 1. Forward\n\nThe token ids go in as a tensor of shape `(4, 1024)`. The embedding table turns each id into a vector of 4,096 numbers, giving `(4, 1024, 4096)`. That tensor passes through every transformer block (attention, then a feed-forward network) and comes out the same shape at the end.\n\n> Figure: One training step, in four parts. Steps: 1. Forward (Ids of shape 4 by 1,024 become logits of shape 4 by 1,024 by 128,256: 525 million numbers, often the largest allocation in the job.); 2. Loss (Cross-entropy against the next token. Positions labelled minus one hundred contribute nothing to the loss and nothing to the gradient.); 3. Backward (One gradient tensor per trainable parameter, the same shape as the parameter, which is why freezing 99 per cent removes 99 per cent of that cost.); 4. Update (AdamW turns gradients into a weight change using two running averages, then the gradients are zeroed.). A batch of four sequences of 1,024 tokens on an 8B model. Every hyperparameter later in the course is a consequence of one of these four lines.\n\nThe last layer projects it to vocabulary size: `(4, 1024, 128256)`. These are the logits, one score per possible next token, at every position.\n\nNotice the size of that tensor. Four times 1,024 times 128,256 is 525 million numbers. In fp32 that is 2.1 GB for a single forward pass, and the backward pass needs room for its gradient too. On many real runs the logits are the largest single allocation in the whole job, bigger than the weights of the layers that produced them. This is why fused or chunked cross-entropy kernels (Liger's, or cut cross-entropy) are worth switching on: they compute the loss without ever materialising the full logits tensor, and they routinely cut peak memory by 30 to 50% on models with large vocabularies.\n\n## 2. Loss\n\nAt every position the model predicted a distribution over the next token, and you know what the next token actually was. Cross-entropy measures the gap: it is the negative log of the probability the model assigned to the correct token.\n\n```python\n# labels are the inputs shifted left by one; -100 marks positions to ignore\nloss = F.cross_entropy(logits[:, :-1].reshape(-1, V),\n                       labels[:, 1:].reshape(-1),\n                       ignore_index=-100)\n```\n\nThat shift by one is the whole self-supervised trick: the target for position *i* is the token at position *i+1*, so a sequence of 1,024 tokens contains 1,023 training examples and you never had to label anything.\n\nThe `ignore_index=-100` is how masking works. Positions labelled -100 contribute nothing to the loss and therefore nothing to the gradient. In instruction tuning that is where you put the prompt tokens, so the model is scored only on what it was supposed to write.\n\n## 3. Backward\n\nThe loss is one number. `loss.backward()` walks the graph in reverse, and for every trainable parameter computes the partial derivative of that number with respect to that parameter. The result is a gradient tensor the same shape as the parameter.\n\nThis is why gradients cost as much memory as the weights they belong to, and why freezing 99% of the model, which is what LoRA does, removes 99% of that cost.\n\nThe backward pass also needs the activations from the forward pass, because the chain rule refers to them. Those activations were kept in memory throughout. That is the third major consumer of memory, and the reason gradient checkpointing exists: throw the activations away, recompute them during the backward pass, pay roughly 30% more compute to save a great deal of memory.\n\n## 4. Update\n\nThe optimiser turns gradients into a change in the weights. In its simplest form:\n\n```\nw = w - learning_rate * gradient\n```\n\nAdamW, which you will almost always use, keeps two running averages per parameter and divides by the square root of the second, so a parameter with consistently small gradients still moves. Those two averages are why AdamW costs 8 bytes per trainable parameter in state alone.\n\nThen the gradients are zeroed and the next batch begins.\n\n## What to take from this\n\nThree facts do most of the work later:\n\n- **The gradient is computed from one batch.** It is a noisy estimate of the direction you want. Larger batches give a less noisy estimate, which is why batch size and learning rate are linked.\n- **Masked positions are invisible.** Not down-weighted, not de-emphasised: absent. If your mask is wrong, the model is learning a different task from the one you think, and the loss curve will look perfectly healthy while it does.\n- **Only trainable parameters get gradients and optimiser state.** Everything about parameter-efficient fine-tuning follows from this one sentence.\n\nThe honest limitation of this picture: it describes what happens, not why it works. Nobody can tell you in advance which weights will change or what the model will have learned when 3,000 of these steps are done. That is why this course spends a whole module on measuring the result rather than predicting it."},"check":{"question":"During a run on an 8B model with a 128k vocabulary, peak memory spikes at the very end of the forward pass, before the optimiser step. What is most likely dominating that spike?","options":[{"text":"The optimiser's two moment buffers, which are allocated lazily on the first forward pass of the run.","correct":false,"tempting":"Misplaces both the timing and the trigger; moment buffers are allocated at the first optimiser step, and they do not spike at the end of a forward pass."},{"text":"The attention key-value cache, which grows with sequence length and is retained across the batch.","correct":false,"tempting":"Imports an inference concept: the KV cache is used for generation, not for a training forward pass, which computes all positions at once."},{"text":"The gradients, which are the same size as the weights and are allocated during the forward pass.","correct":false,"tempting":"Gets the size right but the timing wrong; gradients are populated during the backward pass, after the spike described."},{"text":"The logits tensor (batch times sequence times vocabulary) which can exceed the weights.","correct":true}]},"prev":{"slug":"the-decision-record","title":"Writing the decision down","url":"https://addaly.com/learn/fine-tuning/the-decision-record"},"next":{"slug":"the-loss-you-are-minimising","title":"Reading the loss number","url":"https://addaly.com/learn/fine-tuning/the-loss-you-are-minimising"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'One training step, taken apart', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/one-training-step (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"One training step, taken apart","courseTitle":"Fine-Tuning, and When Not To","module":{"index":2,"slug":"how-training-works","title":"How training actually works"},"answer":null,"keyIdea":"A training step is forward, cross-entropy against the next token, backward to gradients, optimiser update, and the memory, the masking and every hyperparameter you will set are consequences of those four lines.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 2.1","figure":{"type":"flow","steps":[{"label":"Forward","detail":"Ids of shape 4 by 1,024 become logits of shape 4 by 1,024 by 128,256: 525 million numbers, often the largest allocation in the job."},{"label":"Loss","detail":"Cross-entropy against the next token. Positions labelled minus one hundred contribute nothing to the loss and nothing to the gradient."},{"label":"Backward","detail":"One gradient tensor per trainable parameter, the same shape as the parameter, which is why freezing 99 per cent removes 99 per cent of that cost."},{"label":"Update","detail":"AdamW turns gradients into a weight change using two running averages, then the gradients are zeroed."}],"title":"One training step, in four parts","caption":"A batch of four sequences of 1,024 tokens on an 8B model. Every hyperparameter later in the course is a consequence of one of these four lines."},"alt":"Figure: One training step, in four parts. Steps: 1. Forward (Ids of shape 4 by 1,024 become logits of shape 4 by 1,024 by 128,256: 525 million numbers, often the largest allocation in the job.); 2. Loss (Cross-entropy against the next token. Positions labelled minus one hundred contribute nothing to the loss and nothing to the gradient.); 3. Backward (One gradient tensor per trainable parameter, the same shape as the parameter, which is why freezing 99 per cent removes 99 per cent of that cost.); 4. Update (AdamW turns gradients into a weight change using two running averages, then the gradients are zeroed.). A batch of four sequences of 1,024 tokens on an 8B model. Every hyperparameter later in the course is a consequence of one of these four lines."}},"blocks":[{"t":"h2","id":"four-things-happen-in-order","runs":[{"text":"Four things happen, in order"}]},{"t":"p","runs":[{"text":"Everything in this course is a variation on one step, repeated a few thousand times. The step has four parts: a forward pass, a loss, a backward pass, and an update. If you can narrate these with real shapes, every hyperparameter later stops being folklore."}]},{"t":"p","runs":[{"text":"Take a concrete batch: 4 sequences of 1,024 tokens each, on an 8B model with a vocabulary of 128,256."}]},{"t":"h2","id":"1-forward","runs":[{"text":"1. Forward"}]},{"t":"p","runs":[{"text":"The token ids go in as a tensor of shape "},{"text":"(4, 1024)","code":true},{"text":". The embedding table turns each id into a vector of 4,096 numbers, giving "},{"text":"(4, 1024, 4096)","code":true},{"text":". That tensor passes through every transformer block (attention, then a feed-forward network) and comes out the same shape at the end."}]},{"t":"p","runs":[{"text":"The last layer projects it to vocabulary size: "},{"text":"(4, 1024, 128256)","code":true},{"text":". These are the logits, one score per possible next token, at every position."}]},{"t":"p","runs":[{"text":"Notice the size of that tensor. Four times 1,024 times 128,256 is 525 million numbers. In fp32 that is 2.1 GB for a single forward pass, and the backward pass needs room for its gradient too. On many real runs the logits are the largest single allocation in the whole job, bigger than the weights of the layers that produced them. This is why fused or chunked cross-entropy kernels (Liger’s, or cut cross-entropy) are worth switching on: they compute the loss without ever materialising the full logits tensor, and they routinely cut peak memory by 30 to 50% on models with large vocabularies."}]},{"t":"h2","id":"2-loss","runs":[{"text":"2. Loss"}]},{"t":"p","runs":[{"text":"At every position the model predicted a distribution over the next token, and you know what the next token actually was. Cross-entropy measures the gap: it is the negative log of the probability the model assigned to the correct token."}]},{"t":"code","lang":"python","label":"Python","text":"# labels are the inputs shifted left by one; -100 marks positions to ignore\nloss = F.cross_entropy(logits[:, :-1].reshape(-1, V),\n                       labels[:, 1:].reshape(-1),\n                       ignore_index=-100)"},{"t":"p","runs":[{"text":"That shift by one is the whole self-supervised trick: the target for position "},{"text":"i","i":true},{"text":" is the token at position "},{"text":"i+1","i":true},{"text":", so a sequence of 1,024 tokens contains 1,023 training examples and you never had to label anything."}]},{"t":"p","runs":[{"text":"The "},{"text":"ignore_index=-100","code":true},{"text":" is how masking works. Positions labelled -100 contribute nothing to the loss and therefore nothing to the gradient. In instruction tuning that is where you put the prompt tokens, so the model is scored only on what it was supposed to write."}]},{"t":"h2","id":"3-backward","runs":[{"text":"3. Backward"}]},{"t":"p","runs":[{"text":"The loss is one number. "},{"text":"loss.backward()","code":true},{"text":" walks the graph in reverse, and for every trainable parameter computes the partial derivative of that number with respect to that parameter. The result is a gradient tensor the same shape as the parameter."}]},{"t":"p","runs":[{"text":"This is why gradients cost as much memory as the weights they belong to, and why freezing 99% of the model, which is what LoRA does, removes 99% of that cost."}]},{"t":"p","runs":[{"text":"The backward pass also needs the activations from the forward pass, because the chain rule refers to them. Those activations were kept in memory throughout. That is the third major consumer of memory, and the reason gradient checkpointing exists: throw the activations away, recompute them during the backward pass, pay roughly 30% more compute to save a great deal of memory."}]},{"t":"h2","id":"4-update","runs":[{"text":"4. Update"}]},{"t":"p","runs":[{"text":"The optimiser turns gradients into a change in the weights. In its simplest form:"}]},{"t":"code","lang":"","label":"Text","text":"w = w - learning_rate * gradient"},{"t":"p","runs":[{"text":"AdamW, which you will almost always use, keeps two running averages per parameter and divides by the square root of the second, so a parameter with consistently small gradients still moves. Those two averages are why AdamW costs 8 bytes per trainable parameter in state alone."}]},{"t":"p","runs":[{"text":"Then the gradients are zeroed and the next batch begins."}]},{"t":"h2","id":"what-to-take-from-this","runs":[{"text":"What to take from this"}]},{"t":"p","runs":[{"text":"Three facts do most of the work later:"}]},{"t":"ul","items":[[{"text":"The gradient is computed from one batch.","b":true},{"text":" It is a noisy estimate of the direction you want. Larger batches give a less noisy estimate, which is why batch size and learning rate are linked."}],[{"text":"Masked positions are invisible.","b":true},{"text":" Not down-weighted, not de-emphasised: absent. If your mask is wrong, the model is learning a different task from the one you think, and the loss curve will look perfectly healthy while it does."}],[{"text":"Only trainable parameters get gradients and optimiser state.","b":true},{"text":" Everything about parameter-efficient fine-tuning follows from this one sentence."}]]},{"t":"p","runs":[{"text":"The honest limitation of this picture: it describes what happens, not why it works. Nobody can tell you in advance which weights will change or what the model will have learned when 3,000 of these steps are done. That is why this course spends a whole module on measuring the result rather than predicting it."}]}],"check":{"question":[{"text":"During a run on an 8B model with a 128k vocabulary, peak memory spikes at the very end of the forward pass, before the optimiser step. What is most likely dominating that spike?"}],"options":[{"text":[{"text":"The optimiser’s two moment buffers, which are allocated lazily on the first forward pass of the run."}],"correct":false,"tempting":[{"text":"Misplaces both the timing and the trigger; moment buffers are allocated at the first optimiser step, and they do not spike at the end of a forward pass."}]},{"text":[{"text":"The attention key-value cache, which grows with sequence length and is retained across the batch."}],"correct":false,"tempting":[{"text":"Imports an inference concept: the KV cache is used for generation, not for a training forward pass, which computes all positions at once."}]},{"text":[{"text":"The gradients, which are the same size as the weights and are allocated during the forward pass."}],"correct":false,"tempting":[{"text":"Gets the size right but the timing wrong; gradients are populated during the backward pass, after the spike described."}]},{"text":[{"text":"The logits tensor (batch times sequence times vocabulary) which can exceed the weights."}],"correct":true,"tempting":null}],"order":[0,3,2,1],"why":null},"prev":{"slug":"the-decision-record","title":"Writing the decision down","minutes":8},"next":{"slug":"the-loss-you-are-minimising","title":"Reading the loss number","minutes":9}}}