What a model of each size can honestly do

The question everyone asks second

After what should I run comes will it be good enough, and most published answers are either benchmark tables or enthusiasm. Here is a plain account, built from what these sizes reliably do rather than what they can be made to do in a demonstration.

Two mechanisms explain most of the pattern. Stored knowledge scales roughly with parameter count, because facts have to live in the weights. Instruction adherence scales with tuning quality, which is why a well-tuned 4B can follow a format better than a badly tuned 13B. And multi-step reliability compounds: if each step is right 90% of the time, five steps are right 59% of the time, which is why small models look fine on single questions and fall apart in agent loops.

Under 1 billion

Not a chat model, whatever the demo shows. What this size does well, after fine-tuning on your data: classification into a fixed set of labels, intent detection, named entity extraction, spam and toxicity filtering, and reranking. These are real production jobs and a 300M model does them at thousands of items a second on a CPU.

Do not expect it to hold a conversation, follow a multi-part instruction, or know facts.

1 to 3 billion

Code autocomplete. Short structured extraction: pull the date, the amount and the supplier from an invoice. Simple classification without fine-tuning. Translation of short text, with a language-tuned model. Grammar and tone rewriting. On-device assistants with a narrow scope.

The failure to expect: it loses track of instructions with more than two or three clauses, and it confabulates freely when asked anything factual.

3 to 9 billion

The size where most useful local work happens. Reliable extraction into a schema, summarising a page, tagging, rewriting, answering questions from provided context, straightforward tool calls with a well-formed template, and decent conversation within a defined scope.

The characteristic failures are specific enough to plan around. Constraints decay with length: a 4B told to write in British English and avoid a word will comply for two paragraphs and drift by the fourth. Citations get invented when the source is not in front of it. Arithmetic beyond two or three steps is unreliable, and the model will present a wrong result with the same confidence as a right one.

12 to 32 billion

Longer-form writing that holds its shape. Multi-step reasoning that mostly survives. Reasonable code in unfamiliar libraries. Materially better multilingual behaviour, and much better adherence to a long list of constraints. This is where an open model starts to feel like a general assistant rather than a component.

The cost is that you have left laptop territory. A 27B at 4-bit needs roughly 16 GB of weights plus context, which means a well-specified desktop, an Apple machine with a lot of unified memory, or a rented GPU.

70 billion and above

Where open weights compete with commercial APIs on hard tasks: long chains of reasoning, rare knowledge, following a complicated specification, and agent work with many steps. Also where the mixture-of-experts models sit, which compute at the speed of a much smaller model but must still be held in memory in full.

The move that beats going bigger

For most tasks that a 4B is failing, the winning intervention is not a 32B. In rough order of return per hour spent:

1. Narrow the task. Three prompts each doing one thing beat one prompt doing three. A model that fails at classify-and-extract-and-summarise usually succeeds at each separately.
2. Give it the information. Most confabulation is a knowledge problem, and retrieval fixes knowledge problems that no size increase fixes reliably.
3. Constrain the output. Schema-enforced decoding removes an entire class of failure at every size.
4. Show it three examples. Few-shot prompting helps small models far more than large ones, because they have weaker priors about what you want.
5. Fine-tune on 500 examples. A tuned 3B beats an untuned 30B on a narrow task, routinely, and costs less to serve by an order of magnitude.

The reason to know these bands is that they let you say no to the wrong question. Somebody will ask whether a 7B can run their whole support system. The answer is that it can run four pieces of it well and one piece badly, and finding out which is which is an afternoon’s work.