{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"reading-a-model-card","title":"Reading a model card without being sold to","url":"https://addaly.com/learn/running-models-yourself/reading-a-model-card","position":3,"of":82,"minutes":9,"answer":"The Files tab of a model card is verifiable and the prose is a claim: config.json.","keyIdea":"The Files tab of a model card is verifiable and the prose is a claim: config.json, the shard sizes and the chat template tell you more in three minutes than any benchmark table on the page."},"body":{"format":"markdown","text":"## The page is marketing and specification at once\n\nA Hugging Face model card is written by the people who want you to use the model. Some of it is verifiable fact and some is a claim. Learn which fields are which, and you can assess a model in three minutes.\n\n## Read the files, not the prose\n\nThe **Files and versions** tab is the honest half of the page. Four things there tell you most of what you need.\n\n**`config.json`** is the architecture, and it is not editorial:\n\n```json\n{\n  \"architectures\": [\"Qwen3ForCausalLM\"],\n  \"hidden_size\": 4096,\n  \"num_hidden_layers\": 36,\n  \"num_attention_heads\": 32,\n  \"num_key_value_heads\": 8,\n  \"max_position_embeddings\": 40960,\n  \"vocab_size\": 151936,\n  \"torch_dtype\": \"bfloat16\"\n}\n```\n\nEvery number in the memory arithmetic of the next block comes from here. `num_key_value_heads` being much smaller than `num_attention_heads` tells you grouped-query attention is in use and the KV cache will be cheap. `architectures` tells you whether your engine supports it at all: a brand-new architecture name means llama.cpp and vLLM may need an update before the model runs anywhere.\n\n**File sizes.** Add up the `.safetensors` shards. That is the fp16 or bf16 size. Divide by roughly 3.5 for a 4-bit estimate.\n\n**`tokenizer_config.json`** contains `chat_template`, a Jinja string. This is the format the model was trained to expect. It is the single most common cause of poor local output, and it is right there in plain text.\n\n**`generation_config.json`** carries the sampler settings the authors used. Many people never look, then compare the model against a benchmark run at completely different settings.\n\n## The claims, and how to discount them\n\n**Benchmark tables.** Assume the numbers were produced by the model's authors, on their own harness, with prompts they chose. That does not make them false; it makes them incomparable to numbers on another card. Contamination (benchmark items appearing in the training data) is common, rarely tested for, and inflates exactly the scores people quote. Treat published benchmarks as evidence the model is in the right class, never as evidence it is better than another model by two points.\n\n**Context length.** The advertised figure is the maximum position the model will accept, not the length over which it stays accurate. A model advertised at 128k often degrades badly past 16k or 32k on tasks that need real recall. Look for whether the card reports a long-context evaluation at all; most do not.\n\n**\"Outperforms models twice its size.\"** Sometimes true on the specific benchmarks listed and rarely true on your work. This is the claim your own audition exists to test.\n\n## Signals that a model is worth your evening\n\n- A named organisation with previous releases, not an anonymous account.\n- The base model and the fine-tuning data are described, even briefly.\n- A stated knowledge cutoff.\n- Known limitations written by the authors. A card that lists what the model is bad at is a card written by people who measured.\n- Downloads and likes are weak signals but not zero ones: a model with 400,000 downloads has had its template bugs found.\n\n## Signals to be careful of\n\n- No licence field, or a licence that contradicts the base model.\n- Benchmark scores with no harness named and no version.\n- A merge of several models with no evaluation. Merges are cheap to produce and frequently worse than any of their parents on anything that was not measured.\n- Uploaded quants with no note of which base revision they came from. Weights get re-uploaded after bug fixes, and a quant made from the broken revision stays broken forever.\n\n## Do this from the terminal\n\nYou do not need to download 16 GB to inspect a model:\n\n```bash\npip install huggingface_hub\nhf download Qwen/Qwen3-8B config.json tokenizer_config.json --local-dir ./inspect\ncat ./inspect/config.json\n```\n\nThat pulls two small files. From them you can compute the memory sum, see the chat template, and decide whether to spend the bandwidth. On a metered or slow connection this habit is worth more than any benchmark table on the page.\n\n## The three-minute routine\n\nLicence and base model. `config.json` for the architecture and the KV shape. Total `.safetensors` size for the memory estimate. Chat template present and sane. Limitations section written by a human. If all five look right, download it; if two are missing, there is probably a better-documented model in the same class that week."},"check":{"question":"Two 8B models advertise the same 128k context. Reading config.json, model A has 32 layers and 8 key-value heads; model B has 32 layers and 32 key-value heads. What does this predict for running them?","options":[{"text":"Model B will be more accurate at long context, because more key-value heads means more of the context is retained per layer.","correct":false,"tempting":"Treats a memory-layout choice as a quality guarantee; grouped-query attention was adopted because it costs little accuracy, and accuracy at length is not readable from head counts."},{"text":"Model B's KV cache is four times larger per token, so at long context it will need far more memory than model A despite the same parameter count.","correct":true},{"text":"They will behave identically, since KV cache size depends on hidden size and layer count, not on head configuration.","correct":false,"tempting":"Drops the key-value head count from the cache formula, which is the term that differs sixfold between model families."},{"text":"Model A will generate tokens roughly four times faster, because a smaller cache means less memory traffic per weight read during decoding.","correct":false,"tempting":"Confuses cache size with the dominant cost of decoding, which is reading the weights; the cache affects capacity and long-context speed, not the base rate."}]},"prev":{"slug":"the-open-weights-landscape","title":"What \"open\" means, and what it does not","url":"https://addaly.com/learn/running-models-yourself/the-open-weights-landscape"},"next":{"slug":"picking-a-size","title":"What each size class can actually do","url":"https://addaly.com/learn/running-models-yourself/picking-a-size"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Reading a model card without being sold to', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/reading-a-model-card (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Reading a model card without being sold to","courseTitle":"Running Models Yourself","module":{"index":1,"slug":"deciding-to-run-it-yourself","title":"Deciding to run it yourself"},"answer":null,"keyIdea":"The Files tab of a model card is verifiable and the prose is a claim: config.json, the shard sizes and the chat template tell you more in three minutes than any benchmark table on the page.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-page-is-marketing-and-specification-at-once","runs":[{"text":"The page is marketing and specification at once"}]},{"t":"p","runs":[{"text":"A Hugging Face model card is written by the people who want you to use the model. Some of it is verifiable fact and some is a claim. Learn which fields are which, and you can assess a model in three minutes."}]},{"t":"h2","id":"read-the-files-not-the-prose","runs":[{"text":"Read the files, not the prose"}]},{"t":"p","runs":[{"text":"The "},{"text":"Files and versions","b":true},{"text":" tab is the honest half of the page. Four things there tell you most of what you need."}]},{"t":"p","runs":[{"text":"config.json","b":true,"code":true},{"text":" is the architecture, and it is not editorial:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\n  \"architectures\": [\"Qwen3ForCausalLM\"],\n  \"hidden_size\": 4096,\n  \"num_hidden_layers\": 36,\n  \"num_attention_heads\": 32,\n  \"num_key_value_heads\": 8,\n  \"max_position_embeddings\": 40960,\n  \"vocab_size\": 151936,\n  \"torch_dtype\": \"bfloat16\"\n}"},{"t":"p","runs":[{"text":"Every number in the memory arithmetic of the next block comes from here. "},{"text":"num_key_value_heads","code":true},{"text":" being much smaller than "},{"text":"num_attention_heads","code":true},{"text":" tells you grouped-query attention is in use and the KV cache will be cheap. "},{"text":"architectures","code":true},{"text":" tells you whether your engine supports it at all: a brand-new architecture name means llama.cpp and vLLM may need an update before the model runs anywhere."}]},{"t":"p","runs":[{"text":"File sizes.","b":true},{"text":" Add up the "},{"text":".safetensors","code":true},{"text":" shards. That is the fp16 or bf16 size. Divide by roughly 3.5 for a 4-bit estimate."}]},{"t":"p","runs":[{"text":"tokenizer_config.json","b":true,"code":true},{"text":" contains "},{"text":"chat_template","code":true},{"text":", a Jinja string. This is the format the model was trained to expect. It is the single most common cause of poor local output, and it is right there in plain text."}]},{"t":"p","runs":[{"text":"generation_config.json","b":true,"code":true},{"text":" carries the sampler settings the authors used. Many people never look, then compare the model against a benchmark run at completely different settings."}]},{"t":"h2","id":"the-claims-and-how-to-discount-them","runs":[{"text":"The claims, and how to discount them"}]},{"t":"p","runs":[{"text":"Benchmark tables.","b":true},{"text":" Assume the numbers were produced by the model’s authors, on their own harness, with prompts they chose. That does not make them false; it makes them incomparable to numbers on another card. Contamination (benchmark items appearing in the training data) is common, rarely tested for, and inflates exactly the scores people quote. Treat published benchmarks as evidence the model is in the right class, never as evidence it is better than another model by two points."}]},{"t":"p","runs":[{"text":"Context length.","b":true},{"text":" The advertised figure is the maximum position the model will accept, not the length over which it stays accurate. A model advertised at 128k often degrades badly past 16k or 32k on tasks that need real recall. Look for whether the card reports a long-context evaluation at all; most do not."}]},{"t":"p","runs":[{"text":"”Outperforms models twice its size.”","b":true},{"text":" Sometimes true on the specific benchmarks listed and rarely true on your work. This is the claim your own audition exists to test."}]},{"t":"h2","id":"signals-that-a-model-is-worth-your-evening","runs":[{"text":"Signals that a model is worth your evening"}]},{"t":"ul","items":[[{"text":"A named organisation with previous releases, not an anonymous account."}],[{"text":"The base model and the fine-tuning data are described, even briefly."}],[{"text":"A stated knowledge cutoff."}],[{"text":"Known limitations written by the authors. A card that lists what the model is bad at is a card written by people who measured."}],[{"text":"Downloads and likes are weak signals but not zero ones: a model with 400,000 downloads has had its template bugs found."}]]},{"t":"h2","id":"signals-to-be-careful-of","runs":[{"text":"Signals to be careful of"}]},{"t":"ul","items":[[{"text":"No licence field, or a licence that contradicts the base model."}],[{"text":"Benchmark scores with no harness named and no version."}],[{"text":"A merge of several models with no evaluation. Merges are cheap to produce and frequently worse than any of their parents on anything that was not measured."}],[{"text":"Uploaded quants with no note of which base revision they came from. Weights get re-uploaded after bug fixes, and a quant made from the broken revision stays broken forever."}]]},{"t":"h2","id":"do-this-from-the-terminal","runs":[{"text":"Do this from the terminal"}]},{"t":"p","runs":[{"text":"You do not need to download 16 GB to inspect a model:"}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install huggingface_hub\nhf download Qwen/Qwen3-8B config.json tokenizer_config.json --local-dir ./inspect\ncat ./inspect/config.json"},{"t":"p","runs":[{"text":"That pulls two small files. From them you can compute the memory sum, see the chat template, and decide whether to spend the bandwidth. On a metered or slow connection this habit is worth more than any benchmark table on the page."}]},{"t":"h2","id":"the-three-minute-routine","runs":[{"text":"The three-minute routine"}]},{"t":"p","runs":[{"text":"Licence and base model. "},{"text":"config.json","code":true},{"text":" for the architecture and the KV shape. Total "},{"text":".safetensors","code":true},{"text":" size for the memory estimate. Chat template present and sane. Limitations section written by a human. If all five look right, download it; if two are missing, there is probably a better-documented model in the same class that week."}]}],"check":{"question":[{"text":"Two 8B models advertise the same 128k context. Reading config.json, model A has 32 layers and 8 key-value heads; model B has 32 layers and 32 key-value heads. What does this predict for running them?"}],"options":[{"text":[{"text":"Model B will be more accurate at long context, because more key-value heads means more of the context is retained per layer."}],"correct":false,"tempting":[{"text":"Treats a memory-layout choice as a quality guarantee; grouped-query attention was adopted because it costs little accuracy, and accuracy at length is not readable from head counts."}]},{"text":[{"text":"Model B’s KV cache is four times larger per token, so at long context it will need far more memory than model A despite the same parameter count."}],"correct":true,"tempting":null},{"text":[{"text":"They will behave identically, since KV cache size depends on hidden size and layer count, not on head configuration."}],"correct":false,"tempting":[{"text":"Drops the key-value head count from the cache formula, which is the term that differs sixfold between model families."}]},{"text":[{"text":"Model A will generate tokens roughly four times faster, because a smaller cache means less memory traffic per weight read during decoding."}],"correct":false,"tempting":[{"text":"Confuses cache size with the dominant cost of decoding, which is reading the weights; the cache affects capacity and long-context speed, not the base rate."}]}],"order":[0,3,1,2],"why":null},"prev":{"slug":"the-open-weights-landscape","title":"What “open” means, and what it does not","minutes":10},"next":{"slug":"picking-a-size","title":"What each size class can actually do","minutes":10}}}