What is actually inside a model you download

Download one and look

Nothing demystifies this faster than fetching a small open-weights model and listing the folder. A typical one contains:

config.json                 2 KB     the four numbers from the last lesson, and more
model-00001-of-00004.safetensors   4.9 GB   the weights
model-00002-of-00004.safetensors   4.9 GB
model.safetensors.index.json       25 KB    which tensor lives in which shard
tokenizer.json              9 MB     the vocabulary and merge rules
tokenizer_config.json       50 KB    special tokens, and the chat template
generation_config.json      200 B    default temperature, stop tokens

That is the whole artefact. Two observations follow immediately.

There is no training data in there. The 15 trillion tokens the model was trained on are not present, in any form, in a 16 GB file. What survives is a compressed statistical residue. This is worth stating plainly because the question “can I get the training data out” comes up constantly: mostly no, though sequences repeated many times in the corpus can sometimes be reproduced, which is one of the live threads in the copyright disputes covered in module eight.

There is no code in there. Safetensors is a container for arrays and nothing else, deliberately so. The older PyTorch .bin format used Python’s general-purpose object serialisation, which can run arbitrary code at load time and was a genuine supply-chain risk. Prefer .safetensors, and if you must load a .bin from a source you do not fully trust, do it in a throwaway container.

What a tensor looks like

Each weight matrix is stored under a name that maps to a place in the architecture:

model.layers.14.self_attn.q_proj.weight      [4096, 4096]
model.layers.14.self_attn.k_proj.weight      [1024, 4096]
model.layers.14.mlp.gate_proj.weight         [14336, 4096]
model.layers.14.input_layernorm.weight       [4096]
model.embed_tokens.weight                    [128256, 4096]

You can read them yourself in a few lines, without a GPU:

from safetensors import safe_open
with safe_open("model-00001-of-00004.safetensors", framework="pt") as f:
    for k in list(f.keys())[:10]:
        print(k, f.get_slice(k).get_shape())

Print any single number in there and it will mean nothing. That is not a limitation of the tooling. Individual parameters have no interpretation on their own; the interpretable structure, when it exists at all, lives in directions across many of them, which is the subject of module eight.

The chat template is a file, not a rule

Open tokenizer_config.json and you will find a Jinja template that turns a list of messages into one string. Something like:

{% for m in messages %}<|start_header_id|>{{ m.role }}<|end_header_id|>
{{ m.content }}<|eot_id|>{% endfor %}

Use the wrong template with a model and quality collapses in a way that looks like the model being stupid rather than misconfigured. This is the single commonest error when people first run models locally. The library function tokenizer.apply_chat_template(messages) exists precisely so you never hand-roll it.

Formats you will meet

• safetensors: the standard for full-precision weights on Hugging Face.
• GGUF: the format used by llama.cpp and Ollama, which packages weights, tokenizer, chat template and metadata in a single file and supports many quantisation levels. This is the format behind almost every “run it on your laptop” workflow.
• ONNX: a portable graph format used for deployment to phones, browsers and edge devices.

All three are free to use and free to inspect.

What “open” does and does not mean

Most models described as open release the weights and a licence. They generally do not release the training data, the data-filtering code, the training code, or the intermediate checkpoints. That is a meaningful difference from open source as software people use the term, and a few projects (OLMo and Pythia among them) release the whole pipeline precisely to make the distinction visible.

The licences differ too, and some carry conditions on user counts or downstream use. Read the actual licence file before building a product on a model; “open weights” is a description of what you can download, not a statement about what you are permitted to do with it.

Why this lesson exists

People imagine a model as a program with knowledge in it. It is a folder of arrays plus a recipe for how to multiply them. Every capability, every bias and every failure is a property of those numbers, put there by training, and nothing in the folder can be edited by hand to change an answer.