{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"order-balance-and-the-hidden-pattern","title":"What your examples teach by accident","url":"https://addaly.com/learn/prompting/order-balance-and-the-hidden-pattern","position":22,"of":72,"minutes":9,"answer":"Examples teach every property they share (length, tone, punctuation, class proportions, order) so read them as a set and either vary or state anything you did not mean to demonstrate.","keyIdea":"Examples teach every property they share (length, tone, punctuation, class proportions, order) so read them as a set and either vary or state anything you did not mean to demonstrate."},"body":{"format":"markdown","text":"## Examples transmit everything they have in common\n\nYou chose four support messages to demonstrate a classification rule. Without meaning to, you also demonstrated that messages are short, polite, written in English, one paragraph long, and that three out of four are refunds.\n\nThe model has no way of knowing which of those properties you intended. It sees a set and continues the pattern.\n\nThis is the failure that makes people distrust few-shot prompting, and it is entirely preventable once you know to look.\n\n## The four patterns you leak most often\n\n**Class balance.** If four of five examples are labelled `refund`, `refund` becomes the safe guess. Effects like this are well documented: few-shot classification is sensitive to the proportions in the prompt, and to a lesser but real degree to the order of the examples, with the last one carrying extra weight. Keep proportions roughly honest, and if you cannot, say the true distribution in words: \"in reality about 10% are refunds\".\n\n> Figure: How often 'refund' appears. Bars (% of messages labelled refund): In your five examples 80; In real traffic 10; In the model's labels 45. Four refunds among five examples taught that refund is the safe guess, and the model's labels drifted towards the examples rather than towards the traffic. When you cannot show the true proportion, say it in words.\n\n**Length.** Every example output is one sentence, so nothing longer ever appears. Every example is a paragraph, so the two-word answer never comes back even when two words is right. If length should vary, show it varying: one short, one long.\n\n**Politeness and hedging.** Three example replies happen to end with an apology, and now everything ends with an apology, including replies to people who are happy. You never wrote \"apologise\". You demonstrated it three times, which is stronger than writing it.\n\n**Register and language.** If all your examples are formal English, informal or mixed-language inputs get formal English answers. For readers who write in Hinglish, Taglish, or any code-mixed register, this matters a great deal: if you want the reply to match the customer's register, at least one example must do so.\n\n## The audit that takes one minute\n\nRead your examples as a set, ignoring the content, and finish this sentence four times: *all of these are ___.*\n\nAll of these are short. All of these are complaints. All of these end with a question. All of these are about physical products.\n\nEach answer is something you have taught. Keep the ones you meant. For the ones you did not, either vary them or state the exception in words.\n\n## Order, and the recency effect\n\nExamples are not equally weighted. The last example before your real input sits closest to the answer and exerts more pull. Practical consequences:\n\n- **Do not end on your weirdest example.** The exception goes in the middle; a typical case goes last.\n- **Do not group by label.** Three refunds followed by three deliveries teaches a run pattern. Interleave.\n- **If you are testing whether examples help, keep the order fixed** while you change other things, or you will be measuring the shuffle.\n\nNone of these effects are enormous on current models. They are large enough to explain a confusing result, which is when you will want them.\n\n## The label format leaks too\n\nA quieter one. If your examples write labels as `refund`, you will get `refund`. If you write `Refund`, you get `Refund`. If one example writes `Refund.` with a full stop, expect stray full stops in your output column and a downstream script that fails on them.\n\nBe exact and consistent in the mechanical details, because they transfer with perfect fidelity, much better fidelity, in fact, than the reasoning you were trying to teach.\n\n## The rule underneath\n\nEverything in your examples is instruction, whether you meant it or not. Content, length, register, punctuation, order, proportion.\n\nThat is what makes examples powerful and what makes them dangerous. A prose instruction says one thing. A set of examples says everything about itself at once, and you are responsible for all of it.\n\n## A worked correction\n\nA recruitment agency's prompt sorted applications into `interview`, `reject`, `hold`. Five examples: four rejects, one interview, all of them long CVs from candidates with degrees, all written in formal English, all ending with a one-word label and no reason.\n\nEverything about that set was teaching something. The proportions taught that rejection is normal. The uniform length taught that a two-page CV is what a CV looks like, so short ones read as thin. The formal English penalised applicants writing in a second language. And the bare labels taught the model not to explain itself, which removed the one thing that would have made the errors visible.\n\nThe fix was not a better instruction. It was four new examples (two interviews, one hold, one short CV from a strong candidate) a stated real distribution, and a required one-line reason with every label."},"check":{"question":"A prompt classifies feedback as positive, negative or mixed, using six examples: three negative, two positive, one mixed, with the mixed one placed last. Real feedback is about 70% positive. What is the most likely distortion?","options":[{"text":"Mixed will be over-predicted, because it appears last and last examples carry extra weight.","correct":false,"tempting":"Recency is real but secondary here; a single example of a class does not outweigh a proportion that is inverted relative to reality."},{"text":"Negative will be over-predicted, because the example proportions imply that negative feedback is the common case.","correct":true},{"text":"Nothing significant, since six examples are too few to shift a model's behaviour.","correct":false,"tempting":"Few-shot classification is measurably sensitive to class balance even at this size."},{"text":"Positive will be over-predicted, because models are trained to be agreeable and favour positive labels.","correct":false,"tempting":"Borrows sycophancy from a different context; agreeableness in conversation does not translate into a labelling bias here."}]},"prev":{"slug":"choosing-the-awkward-case","title":"The example that earns its place","url":"https://addaly.com/learn/prompting/choosing-the-awkward-case"},"next":{"slug":"rules-and-examples-together","title":"State the rule, then show it","url":"https://addaly.com/learn/prompting/rules-and-examples-together"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What your examples teach by accident', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/order-balance-and-the-hidden-pattern (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What your examples teach by accident","courseTitle":"Getting Real Work out of a Model","module":{"index":3,"slug":"teaching-by-example","title":"Teaching by example"},"answer":null,"keyIdea":"Examples teach every property they share (length, tone, punctuation, class proportions, order) so read them as a set and either vary or state anything you did not mean to demonstrate.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"% of messages labelled refund","title":"How often ‘refund’ appears","series":[{"label":"In your five examples","value":80},{"label":"In real traffic","value":10},{"label":"In the model’s labels","value":45,"highlight":true}],"caption":"Four refunds among five examples taught that refund is the safe guess, and the model’s labels drifted towards the examples rather than towards the traffic. When you cannot show the true proportion, say it in words."},"alt":"Figure: How often ‘refund’ appears. Bars (% of messages labelled refund): In your five examples 80; In real traffic 10; In the model’s labels 45. Four refunds among five examples taught that refund is the safe guess, and the model’s labels drifted towards the examples rather than towards the traffic. When you cannot show the true proportion, say it in words."}},"blocks":[{"t":"h2","id":"examples-transmit-everything-they-have-in-common","runs":[{"text":"Examples transmit everything they have in common"}]},{"t":"p","runs":[{"text":"You chose four support messages to demonstrate a classification rule. Without meaning to, you also demonstrated that messages are short, polite, written in English, one paragraph long, and that three out of four are refunds."}]},{"t":"p","runs":[{"text":"The model has no way of knowing which of those properties you intended. It sees a set and continues the pattern."}]},{"t":"p","runs":[{"text":"This is the failure that makes people distrust few-shot prompting, and it is entirely preventable once you know to look."}]},{"t":"h2","id":"the-four-patterns-you-leak-most-often","runs":[{"text":"The four patterns you leak most often"}]},{"t":"p","runs":[{"text":"Class balance.","b":true},{"text":" If four of five examples are labelled "},{"text":"refund","code":true},{"text":", "},{"text":"refund","code":true},{"text":" becomes the safe guess. Effects like this are well documented: few-shot classification is sensitive to the proportions in the prompt, and to a lesser but real degree to the order of the examples, with the last one carrying extra weight. Keep proportions roughly honest, and if you cannot, say the true distribution in words: “in reality about 10% are refunds”."}]},{"t":"p","runs":[{"text":"Length.","b":true},{"text":" Every example output is one sentence, so nothing longer ever appears. Every example is a paragraph, so the two-word answer never comes back even when two words is right. If length should vary, show it varying: one short, one long."}]},{"t":"p","runs":[{"text":"Politeness and hedging.","b":true},{"text":" Three example replies happen to end with an apology, and now everything ends with an apology, including replies to people who are happy. You never wrote “apologise”. You demonstrated it three times, which is stronger than writing it."}]},{"t":"p","runs":[{"text":"Register and language.","b":true},{"text":" If all your examples are formal English, informal or mixed-language inputs get formal English answers. For readers who write in Hinglish, Taglish, or any code-mixed register, this matters a great deal: if you want the reply to match the customer’s register, at least one example must do so."}]},{"t":"h2","id":"the-audit-that-takes-one-minute","runs":[{"text":"The audit that takes one minute"}]},{"t":"p","runs":[{"text":"Read your examples as a set, ignoring the content, and finish this sentence four times: "},{"text":"all of these are ___.","i":true}]},{"t":"p","runs":[{"text":"All of these are short. All of these are complaints. All of these end with a question. All of these are about physical products."}]},{"t":"p","runs":[{"text":"Each answer is something you have taught. Keep the ones you meant. For the ones you did not, either vary them or state the exception in words."}]},{"t":"h2","id":"order-and-the-recency-effect","runs":[{"text":"Order, and the recency effect"}]},{"t":"p","runs":[{"text":"Examples are not equally weighted. The last example before your real input sits closest to the answer and exerts more pull. Practical consequences:"}]},{"t":"ul","items":[[{"text":"Do not end on your weirdest example.","b":true},{"text":" The exception goes in the middle; a typical case goes last."}],[{"text":"Do not group by label.","b":true},{"text":" Three refunds followed by three deliveries teaches a run pattern. Interleave."}],[{"text":"If you are testing whether examples help, keep the order fixed","b":true},{"text":" while you change other things, or you will be measuring the shuffle."}]]},{"t":"p","runs":[{"text":"None of these effects are enormous on current models. They are large enough to explain a confusing result, which is when you will want them."}]},{"t":"h2","id":"the-label-format-leaks-too","runs":[{"text":"The label format leaks too"}]},{"t":"p","runs":[{"text":"A quieter one. If your examples write labels as "},{"text":"refund","code":true},{"text":", you will get "},{"text":"refund","code":true},{"text":". If you write "},{"text":"Refund","code":true},{"text":", you get "},{"text":"Refund","code":true},{"text":". If one example writes "},{"text":"Refund.","code":true},{"text":" with a full stop, expect stray full stops in your output column and a downstream script that fails on them."}]},{"t":"p","runs":[{"text":"Be exact and consistent in the mechanical details, because they transfer with perfect fidelity, much better fidelity, in fact, than the reasoning you were trying to teach."}]},{"t":"h2","id":"the-rule-underneath","runs":[{"text":"The rule underneath"}]},{"t":"p","runs":[{"text":"Everything in your examples is instruction, whether you meant it or not. Content, length, register, punctuation, order, proportion."}]},{"t":"p","runs":[{"text":"That is what makes examples powerful and what makes them dangerous. A prose instruction says one thing. A set of examples says everything about itself at once, and you are responsible for all of it."}]},{"t":"h2","id":"a-worked-correction","runs":[{"text":"A worked correction"}]},{"t":"p","runs":[{"text":"A recruitment agency’s prompt sorted applications into "},{"text":"interview","code":true},{"text":", "},{"text":"reject","code":true},{"text":", "},{"text":"hold","code":true},{"text":". Five examples: four rejects, one interview, all of them long CVs from candidates with degrees, all written in formal English, all ending with a one-word label and no reason."}]},{"t":"p","runs":[{"text":"Everything about that set was teaching something. The proportions taught that rejection is normal. The uniform length taught that a two-page CV is what a CV looks like, so short ones read as thin. The formal English penalised applicants writing in a second language. And the bare labels taught the model not to explain itself, which removed the one thing that would have made the errors visible."}]},{"t":"p","runs":[{"text":"The fix was not a better instruction. It was four new examples (two interviews, one hold, one short CV from a strong candidate) a stated real distribution, and a required one-line reason with every label."}]}],"check":{"question":[{"text":"A prompt classifies feedback as positive, negative or mixed, using six examples: three negative, two positive, one mixed, with the mixed one placed last. Real feedback is about 70% positive. What is the most likely distortion?"}],"options":[{"text":[{"text":"Mixed will be over-predicted, because it appears last and last examples carry extra weight."}],"correct":false,"tempting":[{"text":"Recency is real but secondary here; a single example of a class does not outweigh a proportion that is inverted relative to reality."}]},{"text":[{"text":"Negative will be over-predicted, because the example proportions imply that negative feedback is the common case."}],"correct":true,"tempting":null},{"text":[{"text":"Nothing significant, since six examples are too few to shift a model’s behaviour."}],"correct":false,"tempting":[{"text":"Few-shot classification is measurably sensitive to class balance even at this size."}]},{"text":[{"text":"Positive will be over-predicted, because models are trained to be agreeable and favour positive labels."}],"correct":false,"tempting":[{"text":"Borrows sycophancy from a different context; agreeableness in conversation does not translate into a labelling bias here."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"choosing-the-awkward-case","title":"The example that earns its place","minutes":8},"next":{"slug":"rules-and-examples-together","title":"State the rule, then show it","minutes":8}}}