What a language model actually is

One function, called repeatedly

Strip away the chat window, the name, the personality and the tools, and a language model is a single mathematical function. It takes a sequence of integers. It returns a list of numbers, one per entry in its vocabulary, saying how likely each entry is to come next. That is the whole thing.

Written out, the shape is this:

logits = model(token_ids)   # in: 1,842 integers.  out: 128,256 numbers

Nothing else goes in. There is no separate database being consulted, no lookup of who you are, no second system deciding whether the answer is true. Every capability the model appears to have (translation, summarising, writing code, refusing a request) has to be expressed through that one output.

Most people’s mental model is a search engine with a friendlier voice. The distance between that picture and the function above is the reason so many behaviours look mysterious. Once you hold the function in mind, a surprising number of “why did it do that” questions answer themselves.

Deterministic in, distribution out

The function itself is deterministic. Feed identical integers to identical weights and you get identical logits, every time. The randomness you experience in a chat comes from what happens next: something has to pick one token from that distribution, and the picking is usually random by design. Lesson by lesson, keep those two apart. The model produces a distribution. The sampler produces a word.

This also means the model does not “choose to answer”. It produces a shape over 128,256 possibilities in which some are heavily favoured. A refusal is a distribution where “I” and “can’t” are near the top. A hallucinated citation is a distribution where a plausible-looking author name is near the top. Same machinery, no internal switch between modes.

What “large” refers to

Four numbers describe almost any model of this kind, and you can find all four on a public model’s config page:

• Vocabulary size: how many distinct tokens exist. Commonly 32,000 to 256,000.
• Model width, usually called hidden_size or d_model: how many numbers represent one token position. 2,048 for a small model, 4,096 for a 7-billion-parameter one, 8,192 and up for the largest.
• Depth: how many identical blocks are stacked. 16 to 32 for small models, 80 or more for the largest.
• Context length: how many tokens fit in one call.

“Large” in large language model refers to the parameter count that falls out of the middle two: the total number of adjustable numbers inside. Seven billion, seventy billion, and upward. Every one of them was set by training, none by hand.

The chat is a costume

You send “Hello”. What actually reaches the function is closer to this:

<|start|>system
You are a helpful assistant. Today is 6 September 2026.
<|end|><|start|>user
Hello<|end|><|start|>assistant

The roles, the system prompt, the boundary markers: all of it is text, converted into the same integers as your message. There is no privileged channel. The model has not been told, in some structural sense, that the system prompt outranks you; it has been trained on data where text after those markers is usually followed by compliant text. That distinction will explain several security properties later in the course.

Base models make it obvious

A model that has only been pretrained, before anyone taught it to behave like an assistant, does not answer questions. Type “What is the capital of Peru?” into a base model and a plausible continuation is another question, because in the wild that string appears most often in quiz lists:

What is the capital of Peru?
What is the capital of Chile?
What is the capital of Bolivia?

It is not being unhelpful. It is doing exactly what it does (continuing text) and the assistant behaviour you are used to was added afterwards, by a training stage covered in module five. You can verify this yourself in a few minutes with any base model on Hugging Face, and it is the single most clarifying thing a newcomer can do.

What to carry forward

The rest of this course is an unpacking of one sentence: tokens in, a distribution over the next token out, repeat. Tokenisation is how the integers are made. Embeddings and attention are how the function computes. Sampling is how a distribution becomes a word. Training is how the weights got their values. Every failure mode in module six is a consequence of that sentence rather than a bug on top of it.