What your examples teach by accident

Examples transmit everything they have in common

You chose four support messages to demonstrate a classification rule. Without meaning to, you also demonstrated that messages are short, polite, written in English, one paragraph long, and that three out of four are refunds.

The model has no way of knowing which of those properties you intended. It sees a set and continues the pattern.

This is the failure that makes people distrust few-shot prompting, and it is entirely preventable once you know to look.

The four patterns you leak most often

Class balance. If four of five examples are labelled refund, refund becomes the safe guess. Effects like this are well documented: few-shot classification is sensitive to the proportions in the prompt, and to a lesser but real degree to the order of the examples, with the last one carrying extra weight. Keep proportions roughly honest, and if you cannot, say the true distribution in words: “in reality about 10% are refunds”.

Length. Every example output is one sentence, so nothing longer ever appears. Every example is a paragraph, so the two-word answer never comes back even when two words is right. If length should vary, show it varying: one short, one long.

Politeness and hedging. Three example replies happen to end with an apology, and now everything ends with an apology, including replies to people who are happy. You never wrote “apologise”. You demonstrated it three times, which is stronger than writing it.

Register and language. If all your examples are formal English, informal or mixed-language inputs get formal English answers. For readers who write in Hinglish, Taglish, or any code-mixed register, this matters a great deal: if you want the reply to match the customer’s register, at least one example must do so.

The audit that takes one minute

Read your examples as a set, ignoring the content, and finish this sentence four times: all of these are ___.

All of these are short. All of these are complaints. All of these end with a question. All of these are about physical products.

Each answer is something you have taught. Keep the ones you meant. For the ones you did not, either vary them or state the exception in words.

Order, and the recency effect

Examples are not equally weighted. The last example before your real input sits closest to the answer and exerts more pull. Practical consequences:

• Do not end on your weirdest example. The exception goes in the middle; a typical case goes last.
• Do not group by label. Three refunds followed by three deliveries teaches a run pattern. Interleave.
• If you are testing whether examples help, keep the order fixed while you change other things, or you will be measuring the shuffle.

None of these effects are enormous on current models. They are large enough to explain a confusing result, which is when you will want them.

The label format leaks too

A quieter one. If your examples write labels as refund, you will get refund. If you write Refund, you get Refund. If one example writes Refund. with a full stop, expect stray full stops in your output column and a downstream script that fails on them.

Be exact and consistent in the mechanical details, because they transfer with perfect fidelity, much better fidelity, in fact, than the reasoning you were trying to teach.

The rule underneath

Everything in your examples is instruction, whether you meant it or not. Content, length, register, punctuation, order, proportion.

That is what makes examples powerful and what makes them dangerous. A prose instruction says one thing. A set of examples says everything about itself at once, and you are responsible for all of it.

A worked correction

A recruitment agency’s prompt sorted applications into interview, reject, hold. Five examples: four rejects, one interview, all of them long CVs from candidates with degrees, all written in formal English, all ending with a one-word label and no reason.

Everything about that set was teaching something. The proportions taught that rejection is normal. The uniform length taught that a two-page CV is what a CV looks like, so short ones read as thin. The formal English penalised applicants writing in a second language. And the bare labels taught the model not to explain itself, which removed the one thing that would have made the errors visible.

The fix was not a better instruction. It was four new examples (two interviews, one hold, one short CV from a strong candidate) a stated real distribution, and a required one-line reason with every label.