{"course":{"slug":"how-llms-work","title":"How a Language Model Actually Works","url":"https://addaly.com/learn/how-llms-work","level":"intermediate","track":"start"},"lesson":{"slug":"what-is-in-a-model-file","title":"What is actually inside a model you download","url":"https://addaly.com/learn/how-llms-work/what-is-in-a-model-file","position":9,"of":78,"minutes":8,"answer":"A model is a folder of numbers plus a configuration and a tokenizer, with no code, no data and no knowledge you can inspect by opening it.","keyIdea":"A model is a folder of numbers plus a configuration and a tokenizer, with no code, no data and no knowledge you can inspect by opening it."},"body":{"format":"markdown","text":"## Download one and look\n\nNothing demystifies this faster than fetching a small open-weights model and listing the folder. A typical one contains:\n\n```text\nconfig.json                 2 KB     the four numbers from the last lesson, and more\nmodel-00001-of-00004.safetensors   4.9 GB   the weights\nmodel-00002-of-00004.safetensors   4.9 GB\nmodel.safetensors.index.json       25 KB    which tensor lives in which shard\ntokenizer.json              9 MB     the vocabulary and merge rules\ntokenizer_config.json       50 KB    special tokens, and the chat template\ngeneration_config.json      200 B    default temperature, stop tokens\n```\n\nThat is the whole artefact. Two observations follow immediately.\n\n**There is no training data in there.** The 15 trillion tokens the model was trained on are not present, in any form, in a 16 GB file. What survives is a compressed statistical residue. This is worth stating plainly because the question \"can I get the training data out\" comes up constantly: mostly no, though sequences repeated many times in the corpus can sometimes be reproduced, which is one of the live threads in the copyright disputes covered in module eight.\n\n**There is no code in there.** Safetensors is a container for arrays and nothing else, deliberately so. The older PyTorch `.bin` format used Python's general-purpose object serialisation, which can run arbitrary code at load time and was a genuine supply-chain risk. Prefer `.safetensors`, and if you must load a `.bin` from a source you do not fully trust, do it in a throwaway container.\n\n## What a tensor looks like\n\nEach weight matrix is stored under a name that maps to a place in the architecture:\n\n```text\nmodel.layers.14.self_attn.q_proj.weight      [4096, 4096]\nmodel.layers.14.self_attn.k_proj.weight      [1024, 4096]\nmodel.layers.14.mlp.gate_proj.weight         [14336, 4096]\nmodel.layers.14.input_layernorm.weight       [4096]\nmodel.embed_tokens.weight                    [128256, 4096]\n```\n\nYou can read them yourself in a few lines, without a GPU:\n\n```python\nfrom safetensors import safe_open\nwith safe_open(\"model-00001-of-00004.safetensors\", framework=\"pt\") as f:\n    for k in list(f.keys())[:10]:\n        print(k, f.get_slice(k).get_shape())\n```\n\nPrint any single number in there and it will mean nothing. That is not a limitation of the tooling. Individual parameters have no interpretation on their own; the interpretable structure, when it exists at all, lives in directions across many of them, which is the subject of module eight.\n\n## The chat template is a file, not a rule\n\nOpen `tokenizer_config.json` and you will find a Jinja template that turns a list of messages into one string. Something like:\n\n```text\n{% for m in messages %}<|start_header_id|>{{ m.role }}<|end_header_id|>\n{{ m.content }}<|eot_id|>{% endfor %}\n```\n\nUse the wrong template with a model and quality collapses in a way that looks like the model being stupid rather than misconfigured. This is the single commonest error when people first run models locally. The library function `tokenizer.apply_chat_template(messages)` exists precisely so you never hand-roll it.\n\n## Formats you will meet\n\n- **safetensors**: the standard for full-precision weights on Hugging Face.\n- **GGUF**: the format used by `llama.cpp` and Ollama, which packages weights, tokenizer, chat template and metadata in a single file and supports many quantisation levels. This is the format behind almost every \"run it on your laptop\" workflow.\n- **ONNX**: a portable graph format used for deployment to phones, browsers and edge devices.\n\nAll three are free to use and free to inspect.\n\n## What \"open\" does and does not mean\n\nMost models described as open release the **weights** and a licence. They generally do not release the training data, the data-filtering code, the training code, or the intermediate checkpoints. That is a meaningful difference from open source as software people use the term, and a few projects (OLMo and Pythia among them) release the whole pipeline precisely to make the distinction visible.\n\nThe licences differ too, and some carry conditions on user counts or downstream use. Read the actual licence file before building a product on a model; \"open weights\" is a description of what you can download, not a statement about what you are permitted to do with it.\n\n## Why this lesson exists\n\nPeople imagine a model as a program with knowledge in it. It is a folder of arrays plus a recipe for how to multiply them. Every capability, every bias and every failure is a property of those numbers, put there by training, and nothing in the folder can be edited by hand to change an answer."},"check":{"question":"A team downloads an open-weights model and finds that its replies are rambling and ignore the system prompt, though the same model behaves well through a hosted API. What is the most likely cause?","options":[{"text":"The hosted version is a larger variant of the same model family.","correct":false,"tempting":"Providers do serve bigger models under similar names, and capability differences are the usual explanation for quality gaps."},{"text":"The local copy was quantised during download and has lost accuracy.","correct":false,"tempting":"Quantisation genuinely costs quality, and downloads of local models often are quantised, so this is a fair suspicion."},{"text":"The weights file is corrupted, since safetensors does not verify contents.","correct":false,"tempting":"Corruption is always possible with multi-gigabyte downloads, which makes an integrity worry reasonable."},{"text":"The messages are not being formatted with the model's own chat template, so the role markers it was trained on are missing.","correct":true}]},"prev":{"slug":"the-shape-of-a-model","title":"Counting the parameters yourself","url":"https://addaly.com/learn/how-llms-work/the-shape-of-a-model"},"next":{"slug":"attention","title":"Attention, without a single matrix","url":"https://addaly.com/learn/how-llms-work/attention"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What is actually inside a model you download', in How a Language Model Actually Works. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-llms-work/what-is-in-a-model-file (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What is actually inside a model you download","courseTitle":"How a Language Model Actually Works","module":{"index":1,"slug":"from-characters-to-vectors","title":"From characters to vectors"},"answer":null,"keyIdea":"A model is a folder of numbers plus a configuration and a tokenizer, with no code, no data and no knowledge you can inspect by opening it.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"download-one-and-look","runs":[{"text":"Download one and look"}]},{"t":"p","runs":[{"text":"Nothing demystifies this faster than fetching a small open-weights model and listing the folder. A typical one contains:"}]},{"t":"code","lang":"text","label":"Text","text":"config.json                 2 KB     the four numbers from the last lesson, and more\nmodel-00001-of-00004.safetensors   4.9 GB   the weights\nmodel-00002-of-00004.safetensors   4.9 GB\nmodel.safetensors.index.json       25 KB    which tensor lives in which shard\ntokenizer.json              9 MB     the vocabulary and merge rules\ntokenizer_config.json       50 KB    special tokens, and the chat template\ngeneration_config.json      200 B    default temperature, stop tokens"},{"t":"p","runs":[{"text":"That is the whole artefact. Two observations follow immediately."}]},{"t":"p","runs":[{"text":"There is no training data in there.","b":true},{"text":" The 15 trillion tokens the model was trained on are not present, in any form, in a 16 GB file. What survives is a compressed statistical residue. This is worth stating plainly because the question “can I get the training data out” comes up constantly: mostly no, though sequences repeated many times in the corpus can sometimes be reproduced, which is one of the live threads in the copyright disputes covered in module eight."}]},{"t":"p","runs":[{"text":"There is no code in there.","b":true},{"text":" Safetensors is a container for arrays and nothing else, deliberately so. The older PyTorch "},{"text":".bin","code":true},{"text":" format used Python’s general-purpose object serialisation, which can run arbitrary code at load time and was a genuine supply-chain risk. Prefer "},{"text":".safetensors","code":true},{"text":", and if you must load a "},{"text":".bin","code":true},{"text":" from a source you do not fully trust, do it in a throwaway container."}]},{"t":"h2","id":"what-a-tensor-looks-like","runs":[{"text":"What a tensor looks like"}]},{"t":"p","runs":[{"text":"Each weight matrix is stored under a name that maps to a place in the architecture:"}]},{"t":"code","lang":"text","label":"Text","text":"model.layers.14.self_attn.q_proj.weight      [4096, 4096]\nmodel.layers.14.self_attn.k_proj.weight      [1024, 4096]\nmodel.layers.14.mlp.gate_proj.weight         [14336, 4096]\nmodel.layers.14.input_layernorm.weight       [4096]\nmodel.embed_tokens.weight                    [128256, 4096]"},{"t":"p","runs":[{"text":"You can read them yourself in a few lines, without a GPU:"}]},{"t":"code","lang":"python","label":"Python","text":"from safetensors import safe_open\nwith safe_open(\"model-00001-of-00004.safetensors\", framework=\"pt\") as f:\n    for k in list(f.keys())[:10]:\n        print(k, f.get_slice(k).get_shape())"},{"t":"p","runs":[{"text":"Print any single number in there and it will mean nothing. That is not a limitation of the tooling. Individual parameters have no interpretation on their own; the interpretable structure, when it exists at all, lives in directions across many of them, which is the subject of module eight."}]},{"t":"h2","id":"the-chat-template-is-a-file-not-a-rule","runs":[{"text":"The chat template is a file, not a rule"}]},{"t":"p","runs":[{"text":"Open "},{"text":"tokenizer_config.json","code":true},{"text":" and you will find a Jinja template that turns a list of messages into one string. Something like:"}]},{"t":"code","lang":"text","label":"Text","text":"{% for m in messages %}<|start_header_id|>{{ m.role }}<|end_header_id|>\n{{ m.content }}<|eot_id|>{% endfor %}"},{"t":"p","runs":[{"text":"Use the wrong template with a model and quality collapses in a way that looks like the model being stupid rather than misconfigured. This is the single commonest error when people first run models locally. The library function "},{"text":"tokenizer.apply_chat_template(messages)","code":true},{"text":" exists precisely so you never hand-roll it."}]},{"t":"h2","id":"formats-you-will-meet","runs":[{"text":"Formats you will meet"}]},{"t":"ul","items":[[{"text":"safetensors","b":true},{"text":": the standard for full-precision weights on Hugging Face."}],[{"text":"GGUF","b":true},{"text":": the format used by "},{"text":"llama.cpp","code":true},{"text":" and Ollama, which packages weights, tokenizer, chat template and metadata in a single file and supports many quantisation levels. This is the format behind almost every “run it on your laptop” workflow."}],[{"text":"ONNX","b":true},{"text":": a portable graph format used for deployment to phones, browsers and edge devices."}]]},{"t":"p","runs":[{"text":"All three are free to use and free to inspect."}]},{"t":"h2","id":"what-open-does-and-does-not-mean","runs":[{"text":"What “open” does and does not mean"}]},{"t":"p","runs":[{"text":"Most models described as open release the "},{"text":"weights","b":true},{"text":" and a licence. They generally do not release the training data, the data-filtering code, the training code, or the intermediate checkpoints. That is a meaningful difference from open source as software people use the term, and a few projects (OLMo and Pythia among them) release the whole pipeline precisely to make the distinction visible."}]},{"t":"p","runs":[{"text":"The licences differ too, and some carry conditions on user counts or downstream use. Read the actual licence file before building a product on a model; “open weights” is a description of what you can download, not a statement about what you are permitted to do with it."}]},{"t":"h2","id":"why-this-lesson-exists","runs":[{"text":"Why this lesson exists"}]},{"t":"p","runs":[{"text":"People imagine a model as a program with knowledge in it. It is a folder of arrays plus a recipe for how to multiply them. Every capability, every bias and every failure is a property of those numbers, put there by training, and nothing in the folder can be edited by hand to change an answer."}]}],"check":{"question":[{"text":"A team downloads an open-weights model and finds that its replies are rambling and ignore the system prompt, though the same model behaves well through a hosted API. What is the most likely cause?"}],"options":[{"text":[{"text":"The hosted version is a larger variant of the same model family."}],"correct":false,"tempting":[{"text":"Providers do serve bigger models under similar names, and capability differences are the usual explanation for quality gaps."}]},{"text":[{"text":"The local copy was quantised during download and has lost accuracy."}],"correct":false,"tempting":[{"text":"Quantisation genuinely costs quality, and downloads of local models often are quantised, so this is a fair suspicion."}]},{"text":[{"text":"The weights file is corrupted, since safetensors does not verify contents."}],"correct":false,"tempting":[{"text":"Corruption is always possible with multi-gigabyte downloads, which makes an integrity worry reasonable."}]},{"text":[{"text":"The messages are not being formatted with the model’s own chat template, so the role markers it was trained on are missing."}],"correct":true,"tempting":null}],"order":[0,2,3,1],"why":null},"prev":{"slug":"the-shape-of-a-model","title":"Counting the parameters yourself","minutes":9},"next":{"slug":"attention","title":"Attention, without a single matrix","minutes":8}}}