How a Language Model Actually Works

The machinery under the chat box, for people who already use these tools daily and want to know what is actually happening inside them. Nine lessons: how your text becomes numbers, what attention really does, what one transformer block adds, why the context window is not memory, what temperature does and does not fix, how the three training stages differ, and why making things up is a property of the method rather than a defect.

1. 1Tokens, and why a token is not a word7 minA token is a chunk of characters chosen by a compression algorithm, not a word and not a letter.
2. 2Embeddings: meaning as a direction7 minAn embedding is not a fixed label on a word; it is a position in space that context rewrites.
3. 3Attention, without a single matrix8 minAttention blends information from earlier positions into the current one; it selects nothing and replaces nothing.
4. 4What one transformer block actually does8 minEvery layer adds to a running total rather than replacing it, and most parameters sit in the feed-forward half.
5. 5Next-token prediction, and where the reasoning comes from8 minReasoning that appears in the output is partly reasoning being done in the output, one token of compute at a time.
6. 6The context window is not memory7 minThe model is stateless; anything it appears to remember was re-sent to it as text this turn.
7. 7Temperature and sampling: what the knob really does6 minTemperature reshapes a distribution the model already produced; it cannot add knowledge the model lacks.
8. 8Pretraining, fine-tuning, and preference training8 minPretraining is where knowledge comes from; the later stages mostly shape behaviour, not facts.
9. 9Why hallucination is a property of the method8 minThe model always outputs a plausible continuation; nothing in the mechanism checks whether it is true.