Getting the model to improve your prompt

It is better at this than people expect

Paste a prompt and ask what is wrong with it, and you often get a useful answer, because “what is missing from this brief” is a pattern-recognition task on a text, which is the sort of thing these systems do well.

It is worth having in the toolkit, and it is worth knowing precisely what it can and cannot see.

The version that works

Vague requests get vague help. “Improve this prompt” produces a longer prompt with a persona bolted on, which is the average of all the prompt advice on the internet, and you now know what most of that advice is worth.

Ask specifically:

Here is my prompt and the output it produced. The output is wrong because [what is wrong]. What information is missing from the prompt that would have prevented this? List only things I could supply; do not suggest phrasings.

Three constraints doing work: you supplied the failure, you asked for missing information rather than better wording, and you excluded phrasing suggestions, which are where the decoration comes from.

Another good form:

Read this prompt as if you had no other context. List every place where a reasonable person could take a different reading from the one I intended, and what each reading would produce.

That is useful, because ambiguity in your own writing is the thing you are least able to see.

What it cannot see

Your context. It cannot know that lead times are always missing from Okonkwo’s quotes, or that your director hates tables. It will suggest generic additions.

Whether its suggestion works. It has no test set and no memory of what happened last time. A suggestion is a hypothesis, and it comes with the same confident tone as a fact.

The output you did not show it. If you describe the failure instead of pasting it, which people do, having learned nothing from module two, the diagnosis is about a category of failure rather than yours.

The trap: infinite polish

You can loop this forever. Ask for improvements, get five, apply them, ask again, get five more. Every round produces suggestions, because a model asked for suggestions produces suggestions. Nothing in the loop tells you when to stop.

Two rules prevent it:

Test each round against your items. Two rounds that produce no measurable improvement means stop.

Cap it at two. In practice the first round finds most of what there is, and the second finds a little. The third is decoration with a helpful tone.

Where it is strong

Finding ambiguity. The reading you did not intend. Very good at this.

Spotting missing fallbacks. It will notice you never said what to do when the document does not state a value.

Spotting contradictions. “You asked for comprehensive and under 100 words.”

Converting vague requirements into checkable ones. “You said ‘professional tone’: do you mean no contractions, no first names, no exclamation marks?” That conversation is worth having and hard to have with yourself.

The self-assessment problem

Do not ask a model to judge whether its own output is good. It is being asked a leading question by the party that produced the answer, in a context containing the answer, by a system trained towards agreement.

If you want a judgement, use a fresh conversation with only the artefact and the criteria, and preferably a different model. That is not superstition: it is removing the two things you know bias the result: the context and the leading question.

The summary

Use it to find what is missing and what is ambiguous. Do not use it to decide whether the prompt is good. That is what the ten items are for, and no amount of articulate self-review substitutes for a count.

The most useful single question

If you take one line from this lesson, take this one, used on a prompt you are about to run over a batch:

Read this prompt and tell me what a careless reader could do that would technically satisfy every instruction while being useless to me.

That question produces the loopholes: the summary that covers every section at one sentence each, the table filled with “varies”, the reply that satisfies the word count by saying nothing. Each loophole it names is one line you can close before the batch runs rather than after, and closing loopholes before a batch is the difference between reviewing forty outputs and rerunning them.