Models that write code

Two different jobs, two different models

Code models split into two kinds that people constantly confuse, and using the wrong one is the most common reason a local coding assistant feels broken.

Completion models finish the code around the cursor. They are trained with fill-in-the-middle, which means the file is cut into prefix, suffix and middle, and the model learns to produce the middle given both sides. The prompt is not a conversation; it is special tokens:

<|fim_prefix|>def parse_date(s):
    <|fim_suffix|>
    return dt<|fim_middle|>

Instruct models answer questions about code and write whole functions from a description. They expect a chat template, and asking one to do inline completion produces an explanation, an apology, and three code fences when what you wanted was four tokens.

If your editor plugin feels wrong, check which of these you have loaded. Every serious code family ships both, and the base or -Base suffixed repository is usually the one with fill-in-the-middle support.

The families

• Qwen Coder. The widest size ladder, from tiny to very large, Apache 2.0 on the main line, with fill-in-the-middle and repository-level training. If you want one answer, this is it.
• DeepSeek Coder. Strong, with a large mixture-of-experts version. The papers explain the repository-level data construction, which is worth reading if you care why these models handle imports across files.
• Codestral (Mistral). Good, and shipped under a non-production licence. It is the standard example of a model that people deploy without reading the terms.
• StarCoder2 (BigCode). Trained on The Stack v2, a corpus built with a public opt-out mechanism for repository owners, which makes it the most defensible provenance story in code. It ships under an OpenRAIL licence with use restrictions rather than a plain open source licence.
• Granite Code (IBM) and CodeGemma (Google) fill out the smaller end.

Sizes

At 1B to 3B you get autocomplete: finishing a line, filling an obvious body, completing a repetitive block. That is a useful product and it runs on a laptop at usable speed, which is the whole reason it exists.

At 7B to 14B you get short functions from a description, test generation, and reasonable explanation of unfamiliar code. Expect to review everything.

Above that, the open models are increasingly competitive at whole-task work (fixing a failing test, making a small change across files) but this is the part of the field that moves fastest, and the honest advice is to measure on your repository rather than trust a number.

The benchmarks are the problem

HumanEval and MBPP are 164 and about 500 short self-contained problems, and both are saturated and heavily contaminated. A model scoring 90 on HumanEval tells you almost nothing about whether it can change your code.

The evaluations worth attention are the ones with an execution harness against real repositories: SWE-bench and its verified subset, LiveCodeBench with its rotating problems, and BigCodeBench for library use. They are harder to fake because something has to actually run.

The evaluation that decides your choice is smaller: take twenty diffs from your own repository history, give the model the surrounding code and the commit message, and see what it produces. An afternoon of that beats every leaderboard, because your repository has conventions no benchmark contains.

Wiring it into an editor, free

The free path here is complete and good.

• Continue is an open source extension for VS Code and JetBrains that points at any OpenAI-compatible endpoint, including a local llama-server or Ollama.
• Tabby is a self-hosted completion server with its own editor plugins.
• llama.vim and llama.vscode talk directly to llama.cpp.
• Aider works from the terminal against a git repository and applies diffs.

A typical laptop setup is a 1.5B to 3B model for inline completion, where latency under about 300 milliseconds is what makes it feel alive, and a larger model called on demand for chat. Run them as two separate endpoints; trying to serve both roles from one model is where the experience falls apart.

The two failures to expect

Imports and versions. Models write against the library versions that dominated their training data. Confident code calling an API that changed two releases ago is the single most common wrong answer, and it looks completely plausible.

Licence contamination in output. Generated code can reproduce distinctive chunks of training data. For anything you ship, the mitigation is the same as for human-written code: review, and for distinctive-looking blocks, search for them.