It is a file, and here is how big

The thing itself

After training finishes, what exists is a file. Not a program in the ordinary sense, not a database of answers, not a copy of the internet sitting on a server. A file, containing a very long list of numbers.

The numbers are called weights, or parameters. Each one is a single value, usually with a few decimal places, and each does nothing on its own. Arranged in the right order and run through the right arithmetic, they produce the behaviour.

Sizes make this concrete. A model described as “8B” has eight billion of these numbers. If each number is stored in sixteen bits (two bytes) the file is about sixteen gigabytes. Store each in four bits instead, a compression called quantisation, and the same model becomes roughly four and a half gigabytes: small enough to sit on a phone, and small enough to run on a laptop with 8GB of memory using free software such as llama.cpp or Ollama. That is a real thing you could do this week, on hardware you probably own.

The biggest commercial models are not published, so their sizes are guesses. The public open-weight ones run from around half a billion parameters up to several hundred billion. A 400B model needs a server, not a laptop.

What is not in the file

Open the file in a text editor and you will find no sentences. There is no article, no page, no lookup table, no index of facts. You cannot search it for “the capital of Kenya” and find a row.

This is worth sitting with, because the natural mental picture (that the model has stored the internet somewhere and fetches from it) leads to wrong predictions about behaviour. If the model held the text, it could quote exactly and would never invent. It does neither. What survives training is the statistical shape of the material, not the material.

A rough comparison: the text used to train a large model runs into the trillions of words, which is many terabytes. The resulting file is a few hundred gigabytes at most. Whatever happened in between, it was not storage.

The honest complication

The clean version above is not quite the whole truth, and the exception is at the centre of the biggest legal fight in this field.

Models do memorise some things word for word. Passages that appeared thousands of times in the training material (a famous poem, a licence header, a common code snippet, a heavily quoted paragraph) can come back verbatim. Researchers have extracted such passages deliberately. It is a small fraction of the material and the models are not designed to do it, but it happens.

So “the model does not contain the text” is true for almost all of the text and false for a little of it. Whether that little makes the model a copy of the works in a legal sense is exactly what courts in several countries are being asked to decide, and they have not agreed. There is a lesson later in this course on that argument. For now, the point is only that the honest statement is more careful than either slogan.

The file does not change

Here is the practical consequence that surprises people most.

While you are talking to a chatbot, the file is not being edited. It is read-only. Ten million people can use the same model at the same time, and each of them is reading the same unchanged numbers. Nothing you type alters it. Nothing you correct is retained in it.

When a chatbot appears to remember your name from an earlier conversation, a separate piece of ordinary software stored that name as text and pasted it back in before your next message. That is a product feature built around the model, not a change inside it. You can usually find it, and delete it, in the settings.

This also explains why a model “gets better” only when the company trains a new one and swaps the file. Improvement arrives in lumps, on their schedule, not gradually from use.

Why this is the useful picture

Hold this image and several things fall into place at once. The model has a fixed cut-off, because the file was finished on a date. It cannot look anything up unless the product wraps it in something that can. It gives similar answers to everybody because everybody is reading the same numbers. It can be copied, which is why an open-weight model, once released, cannot be recalled. And running it costs money every time, because those billions of multiplications happen again for every reply.

All of that follows from the file. Not from intelligence, not from intent: from the fact that the finished product of training is a large, static, unreadable list of numbers that somebody has to run.