{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"prompting-open-models","title":"What changes when nothing is hidden","url":"https://addaly.com/learn/open-models/prompting-open-models","position":43,"of":71,"minutes":9,"answer":"With open weights you supply the scaffolding a hosted product hides, so instructions belong at the end of the prompt, phrased positively, one job per call.","keyIdea":"With open weights you supply the scaffolding a hosted product hides, so instructions belong at the end of the prompt, phrased positively, one job per call, and prompts must be versioned per model because they do not transfer between families."},"body":{"format":"markdown","text":"## You are now the scaffolding\n\nA commercial chat product is not just a model. It is a model plus a long system prompt you never see, plus server-side reformatting, plus a tool harness, plus a safety layer, plus retries. When you download weights, you get the model and none of the rest.\n\nThat is the trade this whole course is about, and in prompting it shows up as a set of specific differences.\n\n**No hidden system prompt.** Whatever behaviour the product had (cite sources, ask clarifying questions, refuse this category, format with headings) was partly instruction, not weights. You have to write it.\n\n**No reformatting.** The exact string you send is what the model sees, special tokens and all. There is no service quietly fixing your template.\n\n**Weaker system-prompt adherence at small sizes.** A 4B model follows a system prompt for a while and then drifts back to its tuned defaults. This is not a bug you can prompt away; it is what a smaller instruction-tuned model does.\n\n## Six things that measurably help\n\n**Put the critical constraint last.** Attention to recent tokens is stronger, and small models in particular follow the end of the prompt more reliably than the middle. If one rule must hold, restate it immediately before the model generates.\n\n**Write positively, not negatively.** *Do not mention the price* frequently produces a mention of the price, because the phrase puts the concept in context and small models are weak at the logical operation of suppression. *Discuss only availability and delivery* works better. This is one of the most reliable differences between prompts that work at 4B and prompts that do not.\n\n**Be literal about the output.** Show the exact shape you want, including an example. Small models copy structure well and infer it badly.\n\n**One job per call.** A prompt that classifies, extracts and summarises will do one of the three well. Three calls to a 4B model are usually cheaper and better than one call to a 30B, and each is separately testable.\n\n**Give a decision procedure, not a description.** *If the message names a product and a defect, label it fault; if it names a payment or a refund, label it billing; if both, prefer billing.* Ordered rules beat adjectives.\n\n**Repeat the format at the end.** A single line (`Reply with only the JSON object.`), placed after the input rather than before it, is worth more than three sentences of instruction at the top.\n\n## Prompts do not transfer\n\nA prompt tuned against one family will underperform on another, sometimes badly. Different instruction-tuning data taught different conventions: how strongly a system prompt binds, whether Markdown headings are natural, how the model handles a numbered list of rules, whether it prefers to explain before answering.\n\nThis has an organisational consequence people resist. **Keep the prompt per model in your repository**, versioned next to the model revision. When you swap models, re-tune and re-measure. Treating the prompt as portable is how a model swap gets blamed for a regression that was a prompt mismatch.\n\n```\nprompts/\n  classify.qwen3-8b.md\n  classify.gemma3-4b.md\n  classify.llama31-8b.md\n```\n\n## Where the system prompt goes when there is none\n\nGemma has no system role. Several other models were tuned with very few system messages and follow them weakly. Read the chat template: if `system` does not appear in the Jinja source, prepend your instructions to the first user message instead, and expect them to bind about as strongly.\n\n## Iterate the way you would test code\n\nFifteen development examples. Change one thing. Run at temperature 0. Count. Keep a log of what you tried and what it scored.\n\nTwo failure patterns to watch for. **Prompt bloat**: a prompt that has grown to 800 tokens through accumulated patches, half of which no longer do anything. Delete a paragraph and re-measure; you will often find the score unchanged and the cost lower. And **overfitting to the fifteen**: hold back another twenty-five examples you touch once at the end, or you have measured your own tuning rather than the prompt.\n\n> The most useful habit: when a prompt fails, print exactly what the model received, including the applied template. About a third of prompting problems turn out to be formatting problems wearing a prompting costume."},"check":{"question":"A prompt containing the line 'do not mention competitors' works on a 32B model but produces competitor mentions on a 4B. What is the best fix?","options":[{"text":"Raise the instruction's priority by moving it into the system prompt and marking it as important.","correct":false,"tempting":"System-prompt adherence is exactly what weakens at small sizes, so this moves the instruction somewhere it binds less."},{"text":"Lower the temperature so the model is less likely to stray into forbidden topics.","correct":false,"tempting":"Temperature affects token choice within the model's distribution; the concept is already activated by the negation."},{"text":"Rewrite it as a positive constraint naming what to discuss, and place it immediately before generation begins.","correct":true},{"text":"Add a post-processing filter that removes competitor names from the output.","correct":false,"tempting":"A reasonable safety net, but it patches the symptom and leaves the model's reasoning shaped around the forbidden concept."}]},"prev":{"slug":"structured-output","title":"Making the output valid by construction","url":"https://addaly.com/learn/open-models/structured-output"},"next":{"slug":"examples-in-the-prompt","title":"Examples, and how to choose them","url":"https://addaly.com/learn/open-models/examples-in-the-prompt"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What changes when nothing is hidden', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/prompting-open-models (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What changes when nothing is hidden","courseTitle":"The Open Model Ecosystem","module":{"index":5,"slug":"getting-good-output","title":"Getting good output"},"answer":null,"keyIdea":"With open weights you supply the scaffolding a hosted product hides, so instructions belong at the end of the prompt, phrased positively, one job per call, and prompts must be versioned per model because they do not transfer between families.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"you-are-now-the-scaffolding","runs":[{"text":"You are now the scaffolding"}]},{"t":"p","runs":[{"text":"A commercial chat product is not just a model. It is a model plus a long system prompt you never see, plus server-side reformatting, plus a tool harness, plus a safety layer, plus retries. When you download weights, you get the model and none of the rest."}]},{"t":"p","runs":[{"text":"That is the trade this whole course is about, and in prompting it shows up as a set of specific differences."}]},{"t":"p","runs":[{"text":"No hidden system prompt.","b":true},{"text":" Whatever behaviour the product had (cite sources, ask clarifying questions, refuse this category, format with headings) was partly instruction, not weights. You have to write it."}]},{"t":"p","runs":[{"text":"No reformatting.","b":true},{"text":" The exact string you send is what the model sees, special tokens and all. There is no service quietly fixing your template."}]},{"t":"p","runs":[{"text":"Weaker system-prompt adherence at small sizes.","b":true},{"text":" A 4B model follows a system prompt for a while and then drifts back to its tuned defaults. This is not a bug you can prompt away; it is what a smaller instruction-tuned model does."}]},{"t":"h2","id":"six-things-that-measurably-help","runs":[{"text":"Six things that measurably help"}]},{"t":"p","runs":[{"text":"Put the critical constraint last.","b":true},{"text":" Attention to recent tokens is stronger, and small models in particular follow the end of the prompt more reliably than the middle. If one rule must hold, restate it immediately before the model generates."}]},{"t":"p","runs":[{"text":"Write positively, not negatively.","b":true},{"text":" "},{"text":"Do not mention the price","i":true},{"text":" frequently produces a mention of the price, because the phrase puts the concept in context and small models are weak at the logical operation of suppression. "},{"text":"Discuss only availability and delivery","i":true},{"text":" works better. This is one of the most reliable differences between prompts that work at 4B and prompts that do not."}]},{"t":"p","runs":[{"text":"Be literal about the output.","b":true},{"text":" Show the exact shape you want, including an example. Small models copy structure well and infer it badly."}]},{"t":"p","runs":[{"text":"One job per call.","b":true},{"text":" A prompt that classifies, extracts and summarises will do one of the three well. Three calls to a 4B model are usually cheaper and better than one call to a 30B, and each is separately testable."}]},{"t":"p","runs":[{"text":"Give a decision procedure, not a description.","b":true},{"text":" "},{"text":"If the message names a product and a defect, label it fault; if it names a payment or a refund, label it billing; if both, prefer billing.","i":true},{"text":" Ordered rules beat adjectives."}]},{"t":"p","runs":[{"text":"Repeat the format at the end.","b":true},{"text":" A single line ("},{"text":"Reply with only the JSON object.","code":true},{"text":"), placed after the input rather than before it, is worth more than three sentences of instruction at the top."}]},{"t":"h2","id":"prompts-do-not-transfer","runs":[{"text":"Prompts do not transfer"}]},{"t":"p","runs":[{"text":"A prompt tuned against one family will underperform on another, sometimes badly. Different instruction-tuning data taught different conventions: how strongly a system prompt binds, whether Markdown headings are natural, how the model handles a numbered list of rules, whether it prefers to explain before answering."}]},{"t":"p","runs":[{"text":"This has an organisational consequence people resist. "},{"text":"Keep the prompt per model in your repository","b":true},{"text":", versioned next to the model revision. When you swap models, re-tune and re-measure. Treating the prompt as portable is how a model swap gets blamed for a regression that was a prompt mismatch."}]},{"t":"code","lang":"","label":"Text","text":"prompts/\n  classify.qwen3-8b.md\n  classify.gemma3-4b.md\n  classify.llama31-8b.md"},{"t":"h2","id":"where-the-system-prompt-goes-when-there-is-none","runs":[{"text":"Where the system prompt goes when there is none"}]},{"t":"p","runs":[{"text":"Gemma has no system role. Several other models were tuned with very few system messages and follow them weakly. Read the chat template: if "},{"text":"system","code":true},{"text":" does not appear in the Jinja source, prepend your instructions to the first user message instead, and expect them to bind about as strongly."}]},{"t":"h2","id":"iterate-the-way-you-would-test-code","runs":[{"text":"Iterate the way you would test code"}]},{"t":"p","runs":[{"text":"Fifteen development examples. Change one thing. Run at temperature 0. Count. Keep a log of what you tried and what it scored."}]},{"t":"p","runs":[{"text":"Two failure patterns to watch for. "},{"text":"Prompt bloat","b":true},{"text":": a prompt that has grown to 800 tokens through accumulated patches, half of which no longer do anything. Delete a paragraph and re-measure; you will often find the score unchanged and the cost lower. And "},{"text":"overfitting to the fifteen","b":true},{"text":": hold back another twenty-five examples you touch once at the end, or you have measured your own tuning rather than the prompt."}]},{"t":"quote","runs":[{"text":"The most useful habit: when a prompt fails, print exactly what the model received, including the applied template. About a third of prompting problems turn out to be formatting problems wearing a prompting costume."}]}],"check":{"question":[{"text":"A prompt containing the line ‘do not mention competitors’ works on a 32B model but produces competitor mentions on a 4B. What is the best fix?"}],"options":[{"text":[{"text":"Raise the instruction’s priority by moving it into the system prompt and marking it as important."}],"correct":false,"tempting":[{"text":"System-prompt adherence is exactly what weakens at small sizes, so this moves the instruction somewhere it binds less."}]},{"text":[{"text":"Lower the temperature so the model is less likely to stray into forbidden topics."}],"correct":false,"tempting":[{"text":"Temperature affects token choice within the model’s distribution; the concept is already activated by the negation."}]},{"text":[{"text":"Rewrite it as a positive constraint naming what to discuss, and place it immediately before generation begins."}],"correct":true,"tempting":null},{"text":[{"text":"Add a post-processing filter that removes competitor names from the output."}],"correct":false,"tempting":[{"text":"A reasonable safety net, but it patches the symptom and leaves the model’s reasoning shaped around the forbidden concept."}]}],"order":[0,3,1,2],"why":null},"prev":{"slug":"structured-output","title":"Making the output valid by construction","minutes":9},"next":{"slug":"examples-in-the-prompt","title":"Examples, and how to choose them","minutes":8}}}