{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"examples-that-teach","title":"Examples that teach, and examples that mislead","url":"https://addaly.com/learn/building-with-ai/examples-that-teach","position":12,"of":80,"minutes":9,"answer":"Choose examples that cover the boundary rather than the typical case, balance the labels, and remember that examples fix shape and consistency but never fix missing knowledge.","keyIdea":"Choose examples that cover the boundary rather than the typical case, balance the labels, and remember that examples fix shape and consistency but never fix missing knowledge."},"body":{"format":"markdown","text":"## Why a few examples beat a paragraph of rules\n\nDescribing a format in prose is lossy. Showing it is not. Three worked examples of the input-output pair you want will usually outperform two hundred words of description, because the model is very good at pattern continuation and only moderately good at following abstract instructions.\n\nThe effect is largest where the task is easy to demonstrate and awkward to define: tone, how much detail counts as enough, where to draw a category boundary, how to handle the messy input you actually receive rather than the clean one you imagined.\n\n```text\nClassify each message as BILLING, TECHNICAL, ACCOUNT or OTHER.\n\nMessage: my card was charged twice this month\nLabel: BILLING\n\nMessage: app crashes when I open the reports tab\nLabel: TECHNICAL\n\nMessage: charged twice AND the app crashed while I was checking\nLabel: BILLING\n\nMessage: {{ input }}\nLabel:\n```\n\nThe third example is the one earning its place. It resolves the ambiguity that would otherwise be resolved differently on every call: when a message spans two categories, billing wins. No amount of prose states that as clearly.\n\n## How many, and which\n\nThe returns curve is steep and then flat. Going from zero examples to three is usually a large improvement. Three to eight is a modest one. Beyond about ten you are mostly buying tokens.\n\nSelect for **coverage of the boundary**, not for typicality. If ninety per cent of your inputs are easy, examples of easy inputs teach almost nothing. Your examples should be the cases you would have to think about:\n\n- The ambiguous case, with the tie-break shown\n- The input that should produce the null or refusal output\n- The malformed input: the empty message, the one word, the wrong language\n- Two cases that look similar and get different labels\n\nThat last pair is the most valuable thing you can put in a prompt, because it defines a boundary by contrast rather than by description.\n\n## Three biases that bite\n\n**Order bias.** Models are sensitive to the sequence of examples, particularly the last one. Put all your BILLING examples together at the end and you will tilt the classifier towards BILLING. Interleave labels, and if a class is rare, do not let it be missing entirely.\n\n**Label distribution bias.** If eight of your ten examples are `APPROVED`, the model infers that approval is the common outcome and approves more. Balance the labels in your examples even when the real world is unbalanced, then handle the real prior in your own thresholds where you can see it.\n\n**Format lock-in, including the mistakes.** The model copies everything, not just the part you meant. Trailing whitespace, inconsistent capitalisation, one example with a full stop and one without: all of it gets reproduced, and inconsistently, which is worse than either. Your examples should be byte-identical in structure.\n\n## Examples are training data you can edit\n\nThe practical loop that separates people who get good at this from people who do not:\n\n1. Run your prompt over a hundred real inputs.\n2. Find the ones it got wrong.\n3. Pick two or three that represent distinct failure *causes*, correct them by hand, and add them as examples.\n4. Re-run and check that nothing that was working broke.\n\nStep four is not optional. Adding an example to fix one failure routinely breaks a different case, because you have shifted the pattern the model is matching. Without a fixed set to re-run, you are playing whack-a-mole with a blindfold on, and most teams do exactly that for months.\n\nWhen the same edit keeps helping, you have discovered a rule. Promote it out of the examples and into the instructions, then delete the example. Instructions generalise; examples anchor. A prompt that grows only in examples becomes expensive and brittle.\n\n## The limitation\n\nExamples teach format, boundary and tone. They do not teach facts, and they do not teach reasoning the model cannot already do.\n\nShowing three examples of correct medical dosages does not make the fourth dosage correct: it makes it correctly formatted. This is the mistake behind a large fraction of disappointing few-shot results: someone adds examples hoping for accuracy and gets consistency instead. If the failure is *the model does not know*, the fix is retrieval or a tool, not another example. If the failure is *the model knows but answers in the wrong shape*, examples are exactly right.\n\nOne cost note. Examples sit in every request, so ten examples at 60 tokens each is 600 tokens on every call forever. Put them in the stable part of your prompt, before the variable material, so prompt caching can absorb them, which is the next lesson but one."},"check":{"question":"A classifier gets the format right but keeps mislabelling messages about failed payments as TECHNICAL rather than BILLING. Which change most directly addresses this?","options":[{"text":"Add a pair of near-identical examples (a failed payment labelled BILLING and an app crash labelled TECHNICAL) so the boundary is defined by contrast.","correct":true},{"text":"Add five more examples of clearly technical messages so that category is better represented.","correct":false,"tempting":"Strengthens a category that is already being over-applied, which pushes the boundary further in the wrong direction."},{"text":"Write a longer definition of each category in the instructions.","correct":false,"tempting":"Adds prose where the failure is a boundary judgement; description is exactly the thing examples are better at than instructions."},{"text":"Switch to a larger model, since the distinction requires more capability.","correct":false,"tempting":"Spends money on a case where the model is not confused about the world but about your labelling convention, which no model can infer."}]},"prev":{"slug":"the-prompt-is-an-interface","title":"A prompt is an interface, not a paragraph","url":"https://addaly.com/learn/building-with-ai/the-prompt-is-an-interface"},"next":{"slug":"decomposition-and-chaining","title":"One big prompt, or four small ones","url":"https://addaly.com/learn/building-with-ai/decomposition-and-chaining"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Examples that teach, and examples that mislead', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/examples-that-teach (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Examples that teach, and examples that mislead","courseTitle":"Building with AI","module":{"index":2,"slug":"prompting-as-engineering","title":"Prompting as engineering"},"answer":null,"keyIdea":"Choose examples that cover the boundary rather than the typical case, balance the labels, and remember that examples fix shape and consistency but never fix missing knowledge.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-a-few-examples-beat-a-paragraph-of-rules","runs":[{"text":"Why a few examples beat a paragraph of rules"}]},{"t":"p","runs":[{"text":"Describing a format in prose is lossy. Showing it is not. Three worked examples of the input-output pair you want will usually outperform two hundred words of description, because the model is very good at pattern continuation and only moderately good at following abstract instructions."}]},{"t":"p","runs":[{"text":"The effect is largest where the task is easy to demonstrate and awkward to define: tone, how much detail counts as enough, where to draw a category boundary, how to handle the messy input you actually receive rather than the clean one you imagined."}]},{"t":"code","lang":"text","label":"Text","text":"Classify each message as BILLING, TECHNICAL, ACCOUNT or OTHER.\n\nMessage: my card was charged twice this month\nLabel: BILLING\n\nMessage: app crashes when I open the reports tab\nLabel: TECHNICAL\n\nMessage: charged twice AND the app crashed while I was checking\nLabel: BILLING\n\nMessage: {{ input }}\nLabel:"},{"t":"p","runs":[{"text":"The third example is the one earning its place. It resolves the ambiguity that would otherwise be resolved differently on every call: when a message spans two categories, billing wins. No amount of prose states that as clearly."}]},{"t":"h2","id":"how-many-and-which","runs":[{"text":"How many, and which"}]},{"t":"p","runs":[{"text":"The returns curve is steep and then flat. Going from zero examples to three is usually a large improvement. Three to eight is a modest one. Beyond about ten you are mostly buying tokens."}]},{"t":"p","runs":[{"text":"Select for "},{"text":"coverage of the boundary","b":true},{"text":", not for typicality. If ninety per cent of your inputs are easy, examples of easy inputs teach almost nothing. Your examples should be the cases you would have to think about:"}]},{"t":"ul","items":[[{"text":"The ambiguous case, with the tie-break shown"}],[{"text":"The input that should produce the null or refusal output"}],[{"text":"The malformed input: the empty message, the one word, the wrong language"}],[{"text":"Two cases that look similar and get different labels"}]]},{"t":"p","runs":[{"text":"That last pair is the most valuable thing you can put in a prompt, because it defines a boundary by contrast rather than by description."}]},{"t":"h2","id":"three-biases-that-bite","runs":[{"text":"Three biases that bite"}]},{"t":"p","runs":[{"text":"Order bias.","b":true},{"text":" Models are sensitive to the sequence of examples, particularly the last one. Put all your BILLING examples together at the end and you will tilt the classifier towards BILLING. Interleave labels, and if a class is rare, do not let it be missing entirely."}]},{"t":"p","runs":[{"text":"Label distribution bias.","b":true},{"text":" If eight of your ten examples are "},{"text":"APPROVED","code":true},{"text":", the model infers that approval is the common outcome and approves more. Balance the labels in your examples even when the real world is unbalanced, then handle the real prior in your own thresholds where you can see it."}]},{"t":"p","runs":[{"text":"Format lock-in, including the mistakes.","b":true},{"text":" The model copies everything, not just the part you meant. Trailing whitespace, inconsistent capitalisation, one example with a full stop and one without: all of it gets reproduced, and inconsistently, which is worse than either. Your examples should be byte-identical in structure."}]},{"t":"h2","id":"examples-are-training-data-you-can-edit","runs":[{"text":"Examples are training data you can edit"}]},{"t":"p","runs":[{"text":"The practical loop that separates people who get good at this from people who do not:"}]},{"t":"ol","start":1,"items":[[{"text":"Run your prompt over a hundred real inputs."}],[{"text":"Find the ones it got wrong."}],[{"text":"Pick two or three that represent distinct failure "},{"text":"causes","i":true},{"text":", correct them by hand, and add them as examples."}],[{"text":"Re-run and check that nothing that was working broke."}]]},{"t":"p","runs":[{"text":"Step four is not optional. Adding an example to fix one failure routinely breaks a different case, because you have shifted the pattern the model is matching. Without a fixed set to re-run, you are playing whack-a-mole with a blindfold on, and most teams do exactly that for months."}]},{"t":"p","runs":[{"text":"When the same edit keeps helping, you have discovered a rule. Promote it out of the examples and into the instructions, then delete the example. Instructions generalise; examples anchor. A prompt that grows only in examples becomes expensive and brittle."}]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"Examples teach format, boundary and tone. They do not teach facts, and they do not teach reasoning the model cannot already do."}]},{"t":"p","runs":[{"text":"Showing three examples of correct medical dosages does not make the fourth dosage correct: it makes it correctly formatted. This is the mistake behind a large fraction of disappointing few-shot results: someone adds examples hoping for accuracy and gets consistency instead. If the failure is "},{"text":"the model does not know","i":true},{"text":", the fix is retrieval or a tool, not another example. If the failure is "},{"text":"the model knows but answers in the wrong shape","i":true},{"text":", examples are exactly right."}]},{"t":"p","runs":[{"text":"One cost note. Examples sit in every request, so ten examples at 60 tokens each is 600 tokens on every call forever. Put them in the stable part of your prompt, before the variable material, so prompt caching can absorb them, which is the next lesson but one."}]}],"check":{"question":[{"text":"A classifier gets the format right but keeps mislabelling messages about failed payments as TECHNICAL rather than BILLING. Which change most directly addresses this?"}],"options":[{"text":[{"text":"Add a pair of near-identical examples (a failed payment labelled BILLING and an app crash labelled TECHNICAL) so the boundary is defined by contrast."}],"correct":true,"tempting":null},{"text":[{"text":"Add five more examples of clearly technical messages so that category is better represented."}],"correct":false,"tempting":[{"text":"Strengthens a category that is already being over-applied, which pushes the boundary further in the wrong direction."}]},{"text":[{"text":"Write a longer definition of each category in the instructions."}],"correct":false,"tempting":[{"text":"Adds prose where the failure is a boundary judgement; description is exactly the thing examples are better at than instructions."}]},{"text":[{"text":"Switch to a larger model, since the distinction requires more capability."}],"correct":false,"tempting":[{"text":"Spends money on a case where the model is not confused about the world but about your labelling convention, which no model can infer."}]}],"order":[1,3,0,2],"why":null},"prev":{"slug":"the-prompt-is-an-interface","title":"A prompt is an interface, not a paragraph","minutes":9},"next":{"slug":"decomposition-and-chaining","title":"One big prompt, or four small ones","minutes":9}}}