What each size class can actually do

Size classes, described by capability

Parameter count is the crudest possible predictor of quality and still the most useful one. Within a generation, and holding training quality roughly constant, here is what each class does. These descriptions age (a good 4B model in 2026 does what an 8B did two years earlier) but the ordering does not.

Under 1B. Classification, sentiment, intent routing, keyword extraction, spelling and grammar repair, simple named-entity work. Runs on a phone. Do not ask it to write anything anyone will read.

1B to 3B. Summarising a paragraph, rewriting for tone, structured extraction from short text, simple question answering with the answer already in the prompt. Reliable at short, well-specified jobs. Loses the thread after about three instructions.

4B to 8B. The workhorse band, and where most local deployments live. Multi-paragraph summaries, decent translation for major languages, single-function code, tagging, drafting, RAG answers over retrieved passages, and reasonable instruction following at up to about eight or ten constraints. Fits on almost any GPU at 4-bit.

12B to 14B. A visible step up in instruction following and in code that spans more than one function. Fits in 12 GB at 4-bit with modest context. The best quality-per-gigabyte point on consumer hardware for many people.

27B to 32B. Approaching useful general assistant behaviour. Handles ambiguity, holds a longer specification, writes code that compiles more often than not. Needs 24 GB at 4-bit and does not leave much room for context.

70B and up. Genuinely capable across most non-frontier work. Needs two 24 GB cards, a 64 GB Mac, or heavy CPU offload and patience.

Mixture-of-experts models break this ranking, which is the next lesson.

Match the class to the job, not to the budget

Most disappointment comes from asking a size class for a capability it does not have, then blaming quantisation or hardware.

Jobs that are size-insensitive (a 3B does them almost as well as a 70B) are worth naming, because these are where local wins outright: classification into a small label set, extracting fields when the fields are present in the text, boilerplate rewriting, translating simple text between high-resource languages, and generating embeddings.

Jobs that are strongly size-sensitive: multi-step reasoning, code across several files, following a fifteen-clause specification exactly, reliable tool calling in a loop, and anything where a plausible wrong answer is worse than no answer.

If your work is entirely in the first list, buy nothing. If it is in the second, the honest question is whether local is the right approach at all.

The cheapest real improvement

Before going up a size class, three things usually gain more:

1. A newer model in the same class. The gap between model generations at fixed size has been larger than the gap between adjacent sizes in the same generation. A current 8B routinely beats a two-year-old 13B.
2. A model trained for your task. A code-specialised 7B beats a general 32B at code completion, and is four times cheaper to run. The same is true for models trained heavily on a particular language.
3. Fixing the prompt and the template. Free, and it is the actual problem more often than people expect.

Multilingual reality

Model families differ enormously here and the card rarely says so plainly. Qwen models are strong across Chinese and reasonable across many Asian languages; Gemma has had broad multilingual training; Llama’s non-English quality varies by language and drops sharply outside its listed set; several European projects target specific language families.

For Indian languages specifically, general models handle Hindi in Devanagari passably at 8B and struggle badly with transliterated Hinglish, with Bengali, Tamil, Telugu and Marathi, and with code-switching mid-sentence. Purpose-built projects exist and are worth searching for by language name rather than assuming the famous model is best. Quantisation hurts these languages first, because they occupy the rarest parts of the weight distribution, so a 4-bit quant that is invisible in English can be clearly worse in Marathi.

Test in the language you will actually use. An English benchmark tells you nothing about this.

A rule for choosing under memory pressure

Given a fixed amount of memory, prefer more parameters at lower precision down to roughly 4 bits, then stop. A 14B at Q4 in 9 GB beats an 8B at Q8 in 9 GB on most work. Below 4 bits the advantage erodes, and by 2 bits a smaller model at a higher bit rate is usually better, unless the model is very large, where even a crushed 70B holds up. That single ordering resolves most “which model for my card” questions without any further argument.