{"course":{"slug":"running-models-yourself","title":"Running Models Yourself","url":"https://addaly.com/learn/running-models-yourself","level":"intermediate","track":"deep"},"lesson":{"slug":"prompting-a-small-model","title":"Prompting a small model is a different craft","url":"https://addaly.com/learn/running-models-yourself/prompting-a-small-model","position":49,"of":82,"minutes":9,"answer":"Small models follow demonstrations better than descriptions and short ordered lists better than paragraphs: three to five instructions per call, format shown by example.","keyIdea":"Small models follow demonstrations better than descriptions and short ordered lists better than paragraphs: three to five instructions per call, format shown by example, and the key constraint repeated at the end where it is acted on."},"body":{"format":"markdown","text":"## What transfers and what does not\n\nAdvice written for frontier models assumes a reader that can hold a long specification, infer what you meant, and recover from an ambiguous instruction. A 4B or 8B model does none of those reliably. The prompting habits that work are different, and they are more like writing a specification than like talking to a colleague.\n\n## Fewer instructions, and they must be sequential\n\nA large model handles fifteen simultaneous constraints. A small one honours the first few and drops the rest, and the ones it drops are usually the ones in the middle.\n\nSo: three to five instructions per call, not fifteen. If you need more, split into two calls. A pipeline of two focused prompts beats one comprehensive prompt at this size, reliably enough to treat as a rule.\n\nWrite them as an ordered list rather than a paragraph. Numbered steps are followed better than prose containing the same information, because the structure survives compression better than the semantics do.\n\n## Show the format instead of describing it\n\nAsking for \"a JSON object with the person's name and their role\" invites variation. Showing one example removes it:\n\n```\nExtract the name and role.\n\nInput: Dr Priya Sharma leads the cardiology unit.\nOutput: {\"name\": \"Priya Sharma\", \"role\": \"head of cardiology\"}\n\nInput: <text>\nOutput:\n```\n\nTwo or three examples are usually the sweet spot for a small model. Beyond about five the returns flatten and you are spending context. Make the examples cover the variation you expect (one straightforward, one awkward, one where a field is missing) because the model will imitate the shape of the examples, including their difficulty.\n\nThis is also the most effective way to handle edge cases. Rather than a rule saying \"if no role is mentioned, use null\", show an example where no role is mentioned and the output has null. Demonstration is followed more reliably than instruction at this size.\n\n## Position matters\n\nInstructions at the very start and the very end of a prompt are followed best; instructions in the middle of a long prompt are followed worst. This holds across model sizes and is pronounced at small ones.\n\nThe practical arrangement: task at the top, then the input, then a short restatement of the required output format at the bottom, immediately before generation begins. Repeating the key constraint at the end is not redundant, it is where the model is most likely to act on it.\n\n## System prompts\n\nA system prompt is a role in the chat template, and how strongly a model attends to it varies enormously by family. Some treat it as near-inviolable; others fold it in with everything else. Two consequences.\n\nKeep it short: a few sentences of stable, task-independent framing. A 600-word system prompt on an 8B is mostly decoration, and it costs prefill on every request.\n\nAnd test whether it works at all. Put a distinctive instruction in the system prompt only (\"always end with the word banana\") and see if it is honoured. If not, the model does not weight the system role much, and your constraints belong in the user message.\n\n## Chain of thought, with a caveat\n\nAsking a small model to work step by step helps on multi-step problems. It also lengthens the output several-fold, which on local hardware is real time.\n\nUse it where the task has dependent steps and skip it where it does not. Classification, extraction and rewriting do not need it: the answer is in the text, and deliberation gives the model room to talk itself out of a correct first instinct. On arithmetic and multi-step reasoning it is worth the tokens.\n\nWhen you do use it and then need structured output, put the reasoning field first in your schema and the answer after it, so the model reasons before committing.\n\n## Things that do not work at this size\n\n- **\"Think carefully\" or \"this is very important\".** Emotional framing has no reliable effect and costs tokens.\n- **\"Do not hallucinate.\"** The model has no access to whether it is hallucinating. Give it the source text instead.\n- **\"You are a world-class expert.\"** Role framing helps style and does not add knowledge the weights do not contain.\n- **Long negative lists.** \"Do not do A, B, C, D\" performs worse than saying what to do. Negations are followed less reliably than positive instructions across the board.\n\n## The method\n\nWrite the prompt, run it against ten items from your audition set at temperature 0, count the failures, change one thing, run again. Twenty minutes of this beats an hour of speculation, and it is the same discipline as everything else in this course: one variable, one measurement, written down."},"check":{"question":"A prompt for an 8B model contains a 400-word specification with twelve numbered requirements. Output honours requirements 1, 2 and 12 but ignores several in the middle. What is the most effective change?","options":[{"text":"Increase the model's temperature so it explores more of the instruction space rather than settling on the first requirements it encounters.","correct":false,"tempting":"Treats instruction following as a sampling problem; higher temperature makes constraint adherence worse, not better."},{"text":"Move the twelve requirements into the system prompt, where the model attends to them more strongly.","correct":false,"tempting":"Assumes a system-role weighting that varies by family and does not change the fact that twelve simultaneous constraints exceed what the model tracks."},{"text":"Split the task into two or three calls of three to five requirements each, since middle-of-prompt instructions are followed worst and the constraint count exceeds what the model tracks.","correct":true},{"text":"Convert the requirements into a JSON schema and use constrained decoding so all twelve are enforced.","correct":false,"tempting":"Enforces shape rather than substance; a grammar can require twelve fields to exist and cannot make their contents honour the requirements."}]},"prev":{"slug":"advanced-samplers","title":"The samplers most interfaces do not show you","url":"https://addaly.com/learn/running-models-yourself/advanced-samplers"},"next":{"slug":"few-shot-and-format-anchoring","title":"Examples, anchoring and getting the shape right","url":"https://addaly.com/learn/running-models-yourself/few-shot-and-format-anchoring"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Prompting a small model is a different craft', in Running Models Yourself. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/running-models-yourself/prompting-a-small-model (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Prompting a small model is a different craft","courseTitle":"Running Models Yourself","module":{"index":6,"slug":"getting-good-output","title":"Getting good output"},"answer":null,"keyIdea":"Small models follow demonstrations better than descriptions and short ordered lists better than paragraphs: three to five instructions per call, format shown by example, and the key constraint repeated at the end where it is acted on.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-transfers-and-what-does-not","runs":[{"text":"What transfers and what does not"}]},{"t":"p","runs":[{"text":"Advice written for frontier models assumes a reader that can hold a long specification, infer what you meant, and recover from an ambiguous instruction. A 4B or 8B model does none of those reliably. The prompting habits that work are different, and they are more like writing a specification than like talking to a colleague."}]},{"t":"h2","id":"fewer-instructions-and-they-must-be-sequential","runs":[{"text":"Fewer instructions, and they must be sequential"}]},{"t":"p","runs":[{"text":"A large model handles fifteen simultaneous constraints. A small one honours the first few and drops the rest, and the ones it drops are usually the ones in the middle."}]},{"t":"p","runs":[{"text":"So: three to five instructions per call, not fifteen. If you need more, split into two calls. A pipeline of two focused prompts beats one comprehensive prompt at this size, reliably enough to treat as a rule."}]},{"t":"p","runs":[{"text":"Write them as an ordered list rather than a paragraph. Numbered steps are followed better than prose containing the same information, because the structure survives compression better than the semantics do."}]},{"t":"h2","id":"show-the-format-instead-of-describing-it","runs":[{"text":"Show the format instead of describing it"}]},{"t":"p","runs":[{"text":"Asking for “a JSON object with the person’s name and their role” invites variation. Showing one example removes it:"}]},{"t":"code","lang":"","label":"Text","text":"Extract the name and role.\n\nInput: Dr Priya Sharma leads the cardiology unit.\nOutput: {\"name\": \"Priya Sharma\", \"role\": \"head of cardiology\"}\n\nInput: <text>\nOutput:"},{"t":"p","runs":[{"text":"Two or three examples are usually the sweet spot for a small model. Beyond about five the returns flatten and you are spending context. Make the examples cover the variation you expect (one straightforward, one awkward, one where a field is missing) because the model will imitate the shape of the examples, including their difficulty."}]},{"t":"p","runs":[{"text":"This is also the most effective way to handle edge cases. Rather than a rule saying “if no role is mentioned, use null”, show an example where no role is mentioned and the output has null. Demonstration is followed more reliably than instruction at this size."}]},{"t":"h2","id":"position-matters","runs":[{"text":"Position matters"}]},{"t":"p","runs":[{"text":"Instructions at the very start and the very end of a prompt are followed best; instructions in the middle of a long prompt are followed worst. This holds across model sizes and is pronounced at small ones."}]},{"t":"p","runs":[{"text":"The practical arrangement: task at the top, then the input, then a short restatement of the required output format at the bottom, immediately before generation begins. Repeating the key constraint at the end is not redundant, it is where the model is most likely to act on it."}]},{"t":"h2","id":"system-prompts","runs":[{"text":"System prompts"}]},{"t":"p","runs":[{"text":"A system prompt is a role in the chat template, and how strongly a model attends to it varies enormously by family. Some treat it as near-inviolable; others fold it in with everything else. Two consequences."}]},{"t":"p","runs":[{"text":"Keep it short: a few sentences of stable, task-independent framing. A 600-word system prompt on an 8B is mostly decoration, and it costs prefill on every request."}]},{"t":"p","runs":[{"text":"And test whether it works at all. Put a distinctive instruction in the system prompt only (“always end with the word banana”) and see if it is honoured. If not, the model does not weight the system role much, and your constraints belong in the user message."}]},{"t":"h2","id":"chain-of-thought-with-a-caveat","runs":[{"text":"Chain of thought, with a caveat"}]},{"t":"p","runs":[{"text":"Asking a small model to work step by step helps on multi-step problems. It also lengthens the output several-fold, which on local hardware is real time."}]},{"t":"p","runs":[{"text":"Use it where the task has dependent steps and skip it where it does not. Classification, extraction and rewriting do not need it: the answer is in the text, and deliberation gives the model room to talk itself out of a correct first instinct. On arithmetic and multi-step reasoning it is worth the tokens."}]},{"t":"p","runs":[{"text":"When you do use it and then need structured output, put the reasoning field first in your schema and the answer after it, so the model reasons before committing."}]},{"t":"h2","id":"things-that-do-not-work-at-this-size","runs":[{"text":"Things that do not work at this size"}]},{"t":"ul","items":[[{"text":"”Think carefully” or “this is very important”.","b":true},{"text":" Emotional framing has no reliable effect and costs tokens."}],[{"text":"”Do not hallucinate.”","b":true},{"text":" The model has no access to whether it is hallucinating. Give it the source text instead."}],[{"text":"”You are a world-class expert.”","b":true},{"text":" Role framing helps style and does not add knowledge the weights do not contain."}],[{"text":"Long negative lists.","b":true},{"text":" “Do not do A, B, C, D” performs worse than saying what to do. Negations are followed less reliably than positive instructions across the board."}]]},{"t":"h2","id":"the-method","runs":[{"text":"The method"}]},{"t":"p","runs":[{"text":"Write the prompt, run it against ten items from your audition set at temperature 0, count the failures, change one thing, run again. Twenty minutes of this beats an hour of speculation, and it is the same discipline as everything else in this course: one variable, one measurement, written down."}]}],"check":{"question":[{"text":"A prompt for an 8B model contains a 400-word specification with twelve numbered requirements. Output honours requirements 1, 2 and 12 but ignores several in the middle. What is the most effective change?"}],"options":[{"text":[{"text":"Increase the model’s temperature so it explores more of the instruction space rather than settling on the first requirements it encounters."}],"correct":false,"tempting":[{"text":"Treats instruction following as a sampling problem; higher temperature makes constraint adherence worse, not better."}]},{"text":[{"text":"Move the twelve requirements into the system prompt, where the model attends to them more strongly."}],"correct":false,"tempting":[{"text":"Assumes a system-role weighting that varies by family and does not change the fact that twelve simultaneous constraints exceed what the model tracks."}]},{"text":[{"text":"Split the task into two or three calls of three to five requirements each, since middle-of-prompt instructions are followed worst and the constraint count exceeds what the model tracks."}],"correct":true,"tempting":null},{"text":[{"text":"Convert the requirements into a JSON schema and use constrained decoding so all twelve are enforced."}],"correct":false,"tempting":[{"text":"Enforces shape rather than substance; a grammar can require twelve fields to exist and cannot make their contents honour the requirements."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"advanced-samplers","title":"The samplers most interfaces do not show you","minutes":9},"next":{"slug":"few-shot-and-format-anchoring","title":"Examples, anchoring and getting the shape right","minutes":8}}}