{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"open-code-models","title":"Models that write code","url":"https://addaly.com/learn/open-models/open-code-models","position":14,"of":71,"minutes":9,"answer":"Completion models trained with fill-in-the-middle and chat-tuned instruct models are different tools with different prompt formats, and code benchmarks like HumanEval are saturated enough that twenty diffs from your own repository are better evidence.","keyIdea":"Completion models trained with fill-in-the-middle and chat-tuned instruct models are different tools with different prompt formats, and code benchmarks like HumanEval are saturated enough that twenty diffs from your own repository are better evidence."},"body":{"format":"markdown","text":"## Two different jobs, two different models\n\nCode models split into two kinds that people constantly confuse, and using the wrong one is the most common reason a local coding assistant feels broken.\n\n**Completion models** finish the code around the cursor. They are trained with fill-in-the-middle, which means the file is cut into prefix, suffix and middle, and the model learns to produce the middle given both sides. The prompt is not a conversation; it is special tokens:\n\n```\n<|fim_prefix|>def parse_date(s):\n    <|fim_suffix|>\n    return dt<|fim_middle|>\n```\n\n**Instruct models** answer questions about code and write whole functions from a description. They expect a chat template, and asking one to do inline completion produces an explanation, an apology, and three code fences when what you wanted was four tokens.\n\nIf your editor plugin feels wrong, check which of these you have loaded. Every serious code family ships both, and the base or `-Base` suffixed repository is usually the one with fill-in-the-middle support.\n\n## The families\n\n- **Qwen Coder.** The widest size ladder, from tiny to very large, Apache 2.0 on the main line, with fill-in-the-middle and repository-level training. If you want one answer, this is it.\n- **DeepSeek Coder.** Strong, with a large mixture-of-experts version. The papers explain the repository-level data construction, which is worth reading if you care why these models handle imports across files.\n- **Codestral (Mistral).** Good, and shipped under a non-production licence. It is the standard example of a model that people deploy without reading the terms.\n- **StarCoder2 (BigCode).** Trained on The Stack v2, a corpus built with a public opt-out mechanism for repository owners, which makes it the most defensible provenance story in code. It ships under an OpenRAIL licence with use restrictions rather than a plain open source licence.\n- **Granite Code (IBM)** and **CodeGemma (Google)** fill out the smaller end.\n\n## Sizes\n\nAt 1B to 3B you get autocomplete: finishing a line, filling an obvious body, completing a repetitive block. That is a useful product and it runs on a laptop at usable speed, which is the whole reason it exists.\n\nAt 7B to 14B you get short functions from a description, test generation, and reasonable explanation of unfamiliar code. Expect to review everything.\n\nAbove that, the open models are increasingly competitive at whole-task work (fixing a failing test, making a small change across files) but this is the part of the field that moves fastest, and the honest advice is to measure on your repository rather than trust a number.\n\n## The benchmarks are the problem\n\nHumanEval and MBPP are 164 and about 500 short self-contained problems, and both are saturated and heavily contaminated. A model scoring 90 on HumanEval tells you almost nothing about whether it can change your code.\n\nThe evaluations worth attention are the ones with an execution harness against real repositories: SWE-bench and its verified subset, LiveCodeBench with its rotating problems, and BigCodeBench for library use. They are harder to fake because something has to actually run.\n\nThe evaluation that decides your choice is smaller: take twenty diffs from your own repository history, give the model the surrounding code and the commit message, and see what it produces. An afternoon of that beats every leaderboard, because your repository has conventions no benchmark contains.\n\n## Wiring it into an editor, free\n\nThe free path here is complete and good.\n\n- **Continue** is an open source extension for VS Code and JetBrains that points at any OpenAI-compatible endpoint, including a local `llama-server` or Ollama.\n- **Tabby** is a self-hosted completion server with its own editor plugins.\n- **llama.vim** and **llama.vscode** talk directly to llama.cpp.\n- **Aider** works from the terminal against a git repository and applies diffs.\n\nA typical laptop setup is a 1.5B to 3B model for inline completion, where latency under about 300 milliseconds is what makes it feel alive, and a larger model called on demand for chat. Run them as two separate endpoints; trying to serve both roles from one model is where the experience falls apart.\n\n## The two failures to expect\n\n**Imports and versions.** Models write against the library versions that dominated their training data. Confident code calling an API that changed two releases ago is the single most common wrong answer, and it looks completely plausible.\n\n**Licence contamination in output.** Generated code can reproduce distinctive chunks of training data. For anything you ship, the mitigation is the same as for human-written code: review, and for distinctive-looking blocks, search for them."},"check":{"question":"A developer loads a 7B instruct code model into an editor plugin for inline autocomplete and finds that every suggestion arrives as a paragraph of explanation followed by a fenced block. What is the underlying cause?","options":[{"text":"The model is too large for low-latency completion, so the plugin is timing out and falling back to chat mode.","correct":false,"tempting":"Latency affects speed, not output shape; a slow completion model still returns a code fragment."},{"text":"The plugin is sending a chat template when it should send a raw prompt, which any model would answer conversationally.","correct":false,"tempting":"Close, but the template is a symptom: an instruct model answers conversationally because that is what it was tuned to do, template or not."},{"text":"Instruct tuning trained the model to respond to a conversational turn, whereas inline completion needs a base model with fill-in-the-middle tokens that emits only the missing span.","correct":true},{"text":"The temperature is too high, so the model is elaborating instead of committing to a completion.","correct":false,"tempting":"Temperature changes which tokens are chosen, not whether the model believes it is in a conversation."}]},"prev":{"slug":"beyond-the-big-six","title":"Regional and specialist models","url":"https://addaly.com/learn/open-models/beyond-the-big-six"},"next":{"slug":"open-embedding-models","title":"Embeddings you can host yourself","url":"https://addaly.com/learn/open-models/open-embedding-models"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Models that write code', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/open-code-models (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Models that write code","courseTitle":"The Open Model Ecosystem","module":{"index":2,"slug":"the-map-of-the-ecosystem","title":"The map, and how to read it"},"answer":null,"keyIdea":"Completion models trained with fill-in-the-middle and chat-tuned instruct models are different tools with different prompt formats, and code benchmarks like HumanEval are saturated enough that twenty diffs from your own repository are better evidence.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-different-jobs-two-different-models","runs":[{"text":"Two different jobs, two different models"}]},{"t":"p","runs":[{"text":"Code models split into two kinds that people constantly confuse, and using the wrong one is the most common reason a local coding assistant feels broken."}]},{"t":"p","runs":[{"text":"Completion models","b":true},{"text":" finish the code around the cursor. They are trained with fill-in-the-middle, which means the file is cut into prefix, suffix and middle, and the model learns to produce the middle given both sides. The prompt is not a conversation; it is special tokens:"}]},{"t":"code","lang":"","label":"Text","text":"<|fim_prefix|>def parse_date(s):\n    <|fim_suffix|>\n    return dt<|fim_middle|>"},{"t":"p","runs":[{"text":"Instruct models","b":true},{"text":" answer questions about code and write whole functions from a description. They expect a chat template, and asking one to do inline completion produces an explanation, an apology, and three code fences when what you wanted was four tokens."}]},{"t":"p","runs":[{"text":"If your editor plugin feels wrong, check which of these you have loaded. Every serious code family ships both, and the base or "},{"text":"-Base","code":true},{"text":" suffixed repository is usually the one with fill-in-the-middle support."}]},{"t":"h2","id":"the-families","runs":[{"text":"The families"}]},{"t":"ul","items":[[{"text":"Qwen Coder.","b":true},{"text":" The widest size ladder, from tiny to very large, Apache 2.0 on the main line, with fill-in-the-middle and repository-level training. If you want one answer, this is it."}],[{"text":"DeepSeek Coder.","b":true},{"text":" Strong, with a large mixture-of-experts version. The papers explain the repository-level data construction, which is worth reading if you care why these models handle imports across files."}],[{"text":"Codestral (Mistral).","b":true},{"text":" Good, and shipped under a non-production licence. It is the standard example of a model that people deploy without reading the terms."}],[{"text":"StarCoder2 (BigCode).","b":true},{"text":" Trained on The Stack v2, a corpus built with a public opt-out mechanism for repository owners, which makes it the most defensible provenance story in code. It ships under an OpenRAIL licence with use restrictions rather than a plain open source licence."}],[{"text":"Granite Code (IBM)","b":true},{"text":" and "},{"text":"CodeGemma (Google)","b":true},{"text":" fill out the smaller end."}]]},{"t":"h2","id":"sizes","runs":[{"text":"Sizes"}]},{"t":"p","runs":[{"text":"At 1B to 3B you get autocomplete: finishing a line, filling an obvious body, completing a repetitive block. That is a useful product and it runs on a laptop at usable speed, which is the whole reason it exists."}]},{"t":"p","runs":[{"text":"At 7B to 14B you get short functions from a description, test generation, and reasonable explanation of unfamiliar code. Expect to review everything."}]},{"t":"p","runs":[{"text":"Above that, the open models are increasingly competitive at whole-task work (fixing a failing test, making a small change across files) but this is the part of the field that moves fastest, and the honest advice is to measure on your repository rather than trust a number."}]},{"t":"h2","id":"the-benchmarks-are-the-problem","runs":[{"text":"The benchmarks are the problem"}]},{"t":"p","runs":[{"text":"HumanEval and MBPP are 164 and about 500 short self-contained problems, and both are saturated and heavily contaminated. A model scoring 90 on HumanEval tells you almost nothing about whether it can change your code."}]},{"t":"p","runs":[{"text":"The evaluations worth attention are the ones with an execution harness against real repositories: SWE-bench and its verified subset, LiveCodeBench with its rotating problems, and BigCodeBench for library use. They are harder to fake because something has to actually run."}]},{"t":"p","runs":[{"text":"The evaluation that decides your choice is smaller: take twenty diffs from your own repository history, give the model the surrounding code and the commit message, and see what it produces. An afternoon of that beats every leaderboard, because your repository has conventions no benchmark contains."}]},{"t":"h2","id":"wiring-it-into-an-editor-free","runs":[{"text":"Wiring it into an editor, free"}]},{"t":"p","runs":[{"text":"The free path here is complete and good."}]},{"t":"ul","items":[[{"text":"Continue","b":true},{"text":" is an open source extension for VS Code and JetBrains that points at any OpenAI-compatible endpoint, including a local "},{"text":"llama-server","code":true},{"text":" or Ollama."}],[{"text":"Tabby","b":true},{"text":" is a self-hosted completion server with its own editor plugins."}],[{"text":"llama.vim","b":true},{"text":" and "},{"text":"llama.vscode","b":true},{"text":" talk directly to llama.cpp."}],[{"text":"Aider","b":true},{"text":" works from the terminal against a git repository and applies diffs."}]]},{"t":"p","runs":[{"text":"A typical laptop setup is a 1.5B to 3B model for inline completion, where latency under about 300 milliseconds is what makes it feel alive, and a larger model called on demand for chat. Run them as two separate endpoints; trying to serve both roles from one model is where the experience falls apart."}]},{"t":"h2","id":"the-two-failures-to-expect","runs":[{"text":"The two failures to expect"}]},{"t":"p","runs":[{"text":"Imports and versions.","b":true},{"text":" Models write against the library versions that dominated their training data. Confident code calling an API that changed two releases ago is the single most common wrong answer, and it looks completely plausible."}]},{"t":"p","runs":[{"text":"Licence contamination in output.","b":true},{"text":" Generated code can reproduce distinctive chunks of training data. For anything you ship, the mitigation is the same as for human-written code: review, and for distinctive-looking blocks, search for them."}]}],"check":{"question":[{"text":"A developer loads a 7B instruct code model into an editor plugin for inline autocomplete and finds that every suggestion arrives as a paragraph of explanation followed by a fenced block. What is the underlying cause?"}],"options":[{"text":[{"text":"The model is too large for low-latency completion, so the plugin is timing out and falling back to chat mode."}],"correct":false,"tempting":[{"text":"Latency affects speed, not output shape; a slow completion model still returns a code fragment."}]},{"text":[{"text":"The plugin is sending a chat template when it should send a raw prompt, which any model would answer conversationally."}],"correct":false,"tempting":[{"text":"Close, but the template is a symptom: an instruct model answers conversationally because that is what it was tuned to do, template or not."}]},{"text":[{"text":"Instruct tuning trained the model to respond to a conversational turn, whereas inline completion needs a base model with fill-in-the-middle tokens that emits only the missing span."}],"correct":true,"tempting":null},{"text":[{"text":"The temperature is too high, so the model is elaborating instead of committing to a completion."}],"correct":false,"tempting":[{"text":"Temperature changes which tokens are chosen, not whether the model believes it is in a conversation."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"beyond-the-big-six","title":"Regional and specialist models","minutes":9},"next":{"slug":"open-embedding-models","title":"Embeddings you can host yourself","minutes":10}}}