{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"what-each-size-can-do","title":"What a model of each size can honestly do","url":"https://addaly.com/learn/open-models/what-each-size-can-do","position":19,"of":71,"minutes":9,"answer":"Stored knowledge scales with parameters and multi-step reliability compounds, so a small model fails on facts and long chains rather than on single narrow tasks, and narrowing the task usually beats increasing the size.","keyIdea":"Stored knowledge scales with parameters and multi-step reliability compounds, so a small model fails on facts and long chains rather than on single narrow tasks, and narrowing the task usually beats increasing the size."},"body":{"format":"markdown","text":"## The question everyone asks second\n\nAfter what should I run comes will it be good enough, and most published answers are either benchmark tables or enthusiasm. Here is a plain account, built from what these sizes reliably do rather than what they can be made to do in a demonstration.\n\nTwo mechanisms explain most of the pattern. **Stored knowledge scales roughly with parameter count**, because facts have to live in the weights. **Instruction adherence scales with tuning quality**, which is why a well-tuned 4B can follow a format better than a badly tuned 13B. And **multi-step reliability compounds**: if each step is right 90% of the time, five steps are right 59% of the time, which is why small models look fine on single questions and fall apart in agent loops.\n\n> Figure: What each size honestly does. Layers, top to bottom: 70B and above (Rare knowledge, long chains, complicated specifications, agent work with many steps. Mixture-of-experts models live here); 12B to 32B (Long-form writing that holds its shape, multi-step reasoning that mostly survives, much better multilingual. No longer a laptop); 3B to 9B (Where most useful local work happens: schema extraction, summarising a page, answers from provided context, one well-formed tool call); 1B to 3B (Code autocomplete, three fields out of an invoice, grammar and tone rewriting, short translation with a language-tuned model); Under 1B, after fine-tuning (Classification into fixed labels, intent detection, entity extraction, spam filtering, reranking. Thousands of items a second on a CPU). Stored knowledge scales with parameter count and multi-step reliability compounds, so a small model fails on facts and long chains rather than on single narrow tasks. For most things a 4B is failing, narrowing the task beats buying a 32B.\n\n## Under 1 billion\n\nNot a chat model, whatever the demo shows. What this size does well, after fine-tuning on your data: classification into a fixed set of labels, intent detection, named entity extraction, spam and toxicity filtering, and reranking. These are real production jobs and a 300M model does them at thousands of items a second on a CPU.\n\nDo not expect it to hold a conversation, follow a multi-part instruction, or know facts.\n\n## 1 to 3 billion\n\nCode autocomplete. Short structured extraction: pull the date, the amount and the supplier from an invoice. Simple classification without fine-tuning. Translation of short text, with a language-tuned model. Grammar and tone rewriting. On-device assistants with a narrow scope.\n\nThe failure to expect: it loses track of instructions with more than two or three clauses, and it confabulates freely when asked anything factual.\n\n## 3 to 9 billion\n\nThe size where most useful local work happens. Reliable extraction into a schema, summarising a page, tagging, rewriting, answering questions from provided context, straightforward tool calls with a well-formed template, and decent conversation within a defined scope.\n\nThe characteristic failures are specific enough to plan around. Constraints decay with length: a 4B told to write in British English and avoid a word will comply for two paragraphs and drift by the fourth. Citations get invented when the source is not in front of it. Arithmetic beyond two or three steps is unreliable, and the model will present a wrong result with the same confidence as a right one.\n\n## 12 to 32 billion\n\nLonger-form writing that holds its shape. Multi-step reasoning that mostly survives. Reasonable code in unfamiliar libraries. Materially better multilingual behaviour, and much better adherence to a long list of constraints. This is where an open model starts to feel like a general assistant rather than a component.\n\nThe cost is that you have left laptop territory. A 27B at 4-bit needs roughly 16 GB of weights plus context, which means a well-specified desktop, an Apple machine with a lot of unified memory, or a rented GPU.\n\n## 70 billion and above\n\nWhere open weights compete with commercial APIs on hard tasks: long chains of reasoning, rare knowledge, following a complicated specification, and agent work with many steps. Also where the mixture-of-experts models sit, which compute at the speed of a much smaller model but must still be held in memory in full.\n\n## The move that beats going bigger\n\nFor most tasks that a 4B is failing, the winning intervention is not a 32B. In rough order of return per hour spent:\n\n1. **Narrow the task.** Three prompts each doing one thing beat one prompt doing three. A model that fails at classify-and-extract-and-summarise usually succeeds at each separately.\n2. **Give it the information.** Most confabulation is a knowledge problem, and retrieval fixes knowledge problems that no size increase fixes reliably.\n3. **Constrain the output.** Schema-enforced decoding removes an entire class of failure at every size.\n4. **Show it three examples.** Few-shot prompting helps small models far more than large ones, because they have weaker priors about what you want.\n5. **Fine-tune on 500 examples.** A tuned 3B beats an untuned 30B on a narrow task, routinely, and costs less to serve by an order of magnitude.\n\n> The reason to know these bands is that they let you say no to the wrong question. Somebody will ask whether a 7B can run their whole support system. The answer is that it can run four pieces of it well and one piece badly, and finding out which is which is an afternoon's work."},"check":{"question":"A 4B model handles one-step support replies well but succeeds only about 55% of the time on a five-step workflow, even though each individual step tests at around 88%. What does this indicate?","options":[{"text":"The model is losing the earlier steps from its context window, so a longer context would fix it.","correct":false,"tempting":"Plausible only if the workflow is long in tokens; five short steps fit easily, and the arithmetic already explains the result."},{"text":"Per-step reliability compounds, so 0.88 to the fifth power is about 0.53; the workflow needs to be split with checks between steps rather than a larger model.","correct":true},{"text":"The workflow prompt is badly written, since a model competent at each step should complete the chain.","correct":false,"tempting":"Assumes step successes combine additively rather than multiplicatively, which is the error the arithmetic exposes."},{"text":"Small models are unsuitable for any multi-step task and the work needs a 70B.","correct":false,"tempting":"A larger model raises per-step accuracy but the compounding remains; at 96% per step a five-step chain is still only 82%."}]},"prev":{"slug":"open-image-and-video-models","title":"Image and video weights, and their licence traps","url":"https://addaly.com/learn/open-models/open-image-and-video-models"},"next":{"slug":"inference-providers","title":"Renting open weights by the token","url":"https://addaly.com/learn/open-models/inference-providers"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What a model of each size can honestly do', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/what-each-size-can-do (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What a model of each size can honestly do","courseTitle":"The Open Model Ecosystem","module":{"index":2,"slug":"the-map-of-the-ecosystem","title":"The map, and how to read it"},"answer":null,"keyIdea":"Stored knowledge scales with parameters and multi-step reliability compounds, so a small model fails on facts and long chains rather than on single narrow tasks, and narrowing the task usually beats increasing the size.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"What each size honestly does","layers":[{"label":"70B and above","detail":"Rare knowledge, long chains, complicated specifications, agent work with many steps. Mixture-of-experts models live here"},{"label":"12B to 32B","detail":"Long-form writing that holds its shape, multi-step reasoning that mostly survives, much better multilingual. No longer a laptop"},{"label":"3B to 9B","detail":"Where most useful local work happens: schema extraction, summarising a page, answers from provided context, one well-formed tool call"},{"label":"1B to 3B","detail":"Code autocomplete, three fields out of an invoice, grammar and tone rewriting, short translation with a language-tuned model"},{"label":"Under 1B, after fine-tuning","detail":"Classification into fixed labels, intent detection, entity extraction, spam filtering, reranking. Thousands of items a second on a CPU"}],"caption":"Stored knowledge scales with parameter count and multi-step reliability compounds, so a small model fails on facts and long chains rather than on single narrow tasks. For most things a 4B is failing, narrowing the task beats buying a 32B."},"alt":"Figure: What each size honestly does. Layers, top to bottom: 70B and above (Rare knowledge, long chains, complicated specifications, agent work with many steps. Mixture-of-experts models live here); 12B to 32B (Long-form writing that holds its shape, multi-step reasoning that mostly survives, much better multilingual. No longer a laptop); 3B to 9B (Where most useful local work happens: schema extraction, summarising a page, answers from provided context, one well-formed tool call); 1B to 3B (Code autocomplete, three fields out of an invoice, grammar and tone rewriting, short translation with a language-tuned model); Under 1B, after fine-tuning (Classification into fixed labels, intent detection, entity extraction, spam filtering, reranking. Thousands of items a second on a CPU). Stored knowledge scales with parameter count and multi-step reliability compounds, so a small model fails on facts and long chains rather than on single narrow tasks. For most things a 4B is failing, narrowing the task beats buying a 32B."}},"blocks":[{"t":"h2","id":"the-question-everyone-asks-second","runs":[{"text":"The question everyone asks second"}]},{"t":"p","runs":[{"text":"After what should I run comes will it be good enough, and most published answers are either benchmark tables or enthusiasm. Here is a plain account, built from what these sizes reliably do rather than what they can be made to do in a demonstration."}]},{"t":"p","runs":[{"text":"Two mechanisms explain most of the pattern. "},{"text":"Stored knowledge scales roughly with parameter count","b":true},{"text":", because facts have to live in the weights. "},{"text":"Instruction adherence scales with tuning quality","b":true},{"text":", which is why a well-tuned 4B can follow a format better than a badly tuned 13B. And "},{"text":"multi-step reliability compounds","b":true},{"text":": if each step is right 90% of the time, five steps are right 59% of the time, which is why small models look fine on single questions and fall apart in agent loops."}]},{"t":"h2","id":"under-1-billion","runs":[{"text":"Under 1 billion"}]},{"t":"p","runs":[{"text":"Not a chat model, whatever the demo shows. What this size does well, after fine-tuning on your data: classification into a fixed set of labels, intent detection, named entity extraction, spam and toxicity filtering, and reranking. These are real production jobs and a 300M model does them at thousands of items a second on a CPU."}]},{"t":"p","runs":[{"text":"Do not expect it to hold a conversation, follow a multi-part instruction, or know facts."}]},{"t":"h2","id":"1-to-3-billion","runs":[{"text":"1 to 3 billion"}]},{"t":"p","runs":[{"text":"Code autocomplete. Short structured extraction: pull the date, the amount and the supplier from an invoice. Simple classification without fine-tuning. Translation of short text, with a language-tuned model. Grammar and tone rewriting. On-device assistants with a narrow scope."}]},{"t":"p","runs":[{"text":"The failure to expect: it loses track of instructions with more than two or three clauses, and it confabulates freely when asked anything factual."}]},{"t":"h2","id":"3-to-9-billion","runs":[{"text":"3 to 9 billion"}]},{"t":"p","runs":[{"text":"The size where most useful local work happens. Reliable extraction into a schema, summarising a page, tagging, rewriting, answering questions from provided context, straightforward tool calls with a well-formed template, and decent conversation within a defined scope."}]},{"t":"p","runs":[{"text":"The characteristic failures are specific enough to plan around. Constraints decay with length: a 4B told to write in British English and avoid a word will comply for two paragraphs and drift by the fourth. Citations get invented when the source is not in front of it. Arithmetic beyond two or three steps is unreliable, and the model will present a wrong result with the same confidence as a right one."}]},{"t":"h2","id":"12-to-32-billion","runs":[{"text":"12 to 32 billion"}]},{"t":"p","runs":[{"text":"Longer-form writing that holds its shape. Multi-step reasoning that mostly survives. Reasonable code in unfamiliar libraries. Materially better multilingual behaviour, and much better adherence to a long list of constraints. This is where an open model starts to feel like a general assistant rather than a component."}]},{"t":"p","runs":[{"text":"The cost is that you have left laptop territory. A 27B at 4-bit needs roughly 16 GB of weights plus context, which means a well-specified desktop, an Apple machine with a lot of unified memory, or a rented GPU."}]},{"t":"h2","id":"70-billion-and-above","runs":[{"text":"70 billion and above"}]},{"t":"p","runs":[{"text":"Where open weights compete with commercial APIs on hard tasks: long chains of reasoning, rare knowledge, following a complicated specification, and agent work with many steps. Also where the mixture-of-experts models sit, which compute at the speed of a much smaller model but must still be held in memory in full."}]},{"t":"h2","id":"the-move-that-beats-going-bigger","runs":[{"text":"The move that beats going bigger"}]},{"t":"p","runs":[{"text":"For most tasks that a 4B is failing, the winning intervention is not a 32B. In rough order of return per hour spent:"}]},{"t":"ol","start":1,"items":[[{"text":"Narrow the task.","b":true},{"text":" Three prompts each doing one thing beat one prompt doing three. A model that fails at classify-and-extract-and-summarise usually succeeds at each separately."}],[{"text":"Give it the information.","b":true},{"text":" Most confabulation is a knowledge problem, and retrieval fixes knowledge problems that no size increase fixes reliably."}],[{"text":"Constrain the output.","b":true},{"text":" Schema-enforced decoding removes an entire class of failure at every size."}],[{"text":"Show it three examples.","b":true},{"text":" Few-shot prompting helps small models far more than large ones, because they have weaker priors about what you want."}],[{"text":"Fine-tune on 500 examples.","b":true},{"text":" A tuned 3B beats an untuned 30B on a narrow task, routinely, and costs less to serve by an order of magnitude."}]]},{"t":"quote","runs":[{"text":"The reason to know these bands is that they let you say no to the wrong question. Somebody will ask whether a 7B can run their whole support system. The answer is that it can run four pieces of it well and one piece badly, and finding out which is which is an afternoon’s work."}]}],"check":{"question":[{"text":"A 4B model handles one-step support replies well but succeeds only about 55% of the time on a five-step workflow, even though each individual step tests at around 88%. What does this indicate?"}],"options":[{"text":[{"text":"The model is losing the earlier steps from its context window, so a longer context would fix it."}],"correct":false,"tempting":[{"text":"Plausible only if the workflow is long in tokens; five short steps fit easily, and the arithmetic already explains the result."}]},{"text":[{"text":"Per-step reliability compounds, so 0.88 to the fifth power is about 0.53; the workflow needs to be split with checks between steps rather than a larger model."}],"correct":true,"tempting":null},{"text":[{"text":"The workflow prompt is badly written, since a model competent at each step should complete the chain."}],"correct":false,"tempting":[{"text":"Assumes step successes combine additively rather than multiplicatively, which is the error the arithmetic exposes."}]},{"text":[{"text":"Small models are unsuitable for any multi-step task and the work needs a 70B."}],"correct":false,"tempting":[{"text":"A larger model raises per-step accuracy but the compounding remains; at 96% per step a five-step chain is still only 82%."}]}],"order":[0,3,2,1],"why":null},"prev":{"slug":"open-image-and-video-models","title":"Image and video weights, and their licence traps","minutes":9},"next":{"slug":"inference-providers","title":"Renting open weights by the token","minutes":8}}}