The example that earns its place

Obvious examples teach nothing

Here is a set somebody wrote to classify support messages:

"Where is my parcel?"            → delivery
"My parcel hasn't arrived."      → delivery
"I still haven't got my order."  → delivery
"Can you refund me?"             → refund

Four examples, three of which are the same example. The model already knew that “where is my parcel” is about delivery; it did not need three demonstrations of the easy case, and it received no information about anything that is actually hard.

An example is only worth its space if a competent reader could plausibly have got it wrong.

Where the information lives

Think about your task and find the places where two reasonable people would disagree. That is where your examples go.

For a support classifier, the disagreements are:

• The parcel arrived broken and the customer wants money. Delivery or refund?
• The customer is angry but is asking a normal question. Complaint or query?
• The message is in two languages and half of it is about something else.
• The message says “cancel my order” but it has already shipped.

Four examples, each resolving one genuine ambiguity. That set carries roughly four times the information of the first one, in the same space.

The three examples worth having

If you only have room for a handful, pick these:

1. The boundary case. The one that sits closest to the line between two answers. Include the label and, where it helps, a one-line reason: → refund (classify by what they are asking for, not what went wrong). A stated reason generalises to cases you did not show.

2. The case you keep getting wrong. Go through the outputs that annoyed you. The failure you have corrected three times is a failure your prompt does not cover. Turn it into an example: the input that produced the bad output, plus the output you wanted.

3. The one that stops the over-produced answer. Every classifier has a default it reaches for. If everything ambiguous is coming back as other, include an ambiguous case labelled something else, so other stops looking like the safe choice.

Where to find real awkward cases

Not by imagining them. Imagined edge cases are systematically wrong, because you imagine the ones you have already thought about, which are the ones your instructions already cover.

Take fifty real items: real messages, real invoices, real rows. Label them yourself, quickly. The ones where you hesitated for more than two seconds are your example set. Your hesitation is the measurement.

This takes twenty minutes and it is the single highest-value twenty minutes in this course. It also produces something else you will want later: a small set of items with known answers, which is a test set, and lets you check whether any change to your prompt actually helped.

What to do when you cannot decide either

Sometimes you hesitate because the rule does not exist yet. The broken-parcel-refund case has no answer until somebody decides.

That is not a prompting problem. It is a policy gap that your prompt has revealed, and the right response is to make the decision, write it down in one line, and put it in the prompt as a rule with an example attached. Teams often discover in this exact moment that two people have been handling the case differently for a year.

The maintenance

An awkward case stops being awkward once your prompt handles it. It does not stop being useful (it is what keeps handling it), but you should review the set when something changes: a new product line, a new policy, a new failure that keeps appearing.

Four to six examples, each earning its place, reviewed every few months. That is a working example set, and it will outperform any twenty-example collection assembled from cases nobody was unsure about.

One warning about hard cases

A set made entirely of edge cases has its own distortion: it implies that edge cases are normal. If four out of four examples are awkward, ordinary items start attracting exotic treatment, and you will see the model hedging on a message that needed one word.

So the working shape is mostly-hard-with-an-anchor: three or four boundary cases and one plainly typical one, placed last. The typical example costs you almost nothing and it re-establishes what the ordinary case looks like right before the model answers.