The decision the number has to serve

“Does it work” is not a question

A question is answerable when a specific answer would change what you do next. Does the assistant work? fails that test. Two people can read the same fifty outputs and disagree completely, because they are quietly answering different questions: one is asking whether this would embarrass us in public, the other whether it saves the support team an hour a day. Both are reasonable. Neither is the same measurement.

So before you pick a metric, write three lines. They take ten minutes and they save weeks.

The three lines

1. The decision. What will you actually do differently depending on the result? Ship or hold. Route all tickets through it or one in ten. Replace the human first draft, or keep the human and let the model suggest. If no decision hangs on the number, you are collecting it for decoration and you can stop now.

2. The threshold. What result would make you ship, and what result would make you not? State it as a number before you see one. “We ship if it pulls the correct amount and date from at least 90 of 100 invoices.” A threshold written after the run is a rationalisation; every number looks like support for what you already wanted. A threshold written before is a commitment you can be held to, including by yourself at midnight.

3. The unacceptable failure. Most systems have one error that cannot be traded against anything else. A pharmacy assistant may not invent a dose. A refund bot may not promise money the policy does not allow. This does not belong in the average. It is a gate: score it separately, and let a single occurrence in a hundred fail the run, if that is the truth.

Three lines, dated, committed next to the eval set. That file is the closest thing applied AI has to a pre-registration, and it is the cheapest honesty device in this course.

A worked example

A clinic in Pune wants a WhatsApp bot that handles appointment requests. “Does it work” is unanswerable. Here is the same intention, written so it can be measured:

Decision: whether the bot answers messages directly outside office hours, or only drafts a reply for the receptionist to send in the morning.

Threshold: on 100 real messages from last month, it extracts the requested doctor, date and time correctly in at least 92, and asks a clarifying question rather than guessing whenever any of the three is missing.

Unacceptable: naming a doctor who does not work at the clinic, or confirming a slot that is not free. Zero tolerated in 100.

Notice what changed. The vague version invited a demo. This version tells you what to collect, how many, what to label, and what result ends the argument.

Metrics that look like evidence and are not

Three keep appearing in decks:

• Average user rating. Only a small, self-selected minority ever clicks a rating (in one published developer assistant, 2% of conversations got one), and the people who click are not a random sample. A 4.6 average tells you about the people who rate, not the people who use.
• Tokens, sessions, messages. Usage is not quality. A support bot whose message count doubled may be failing twice as often and being asked again.
• A model’s own confidence. Ask a language model how sure it is and it will usually give a round number between 80% and 100%, often while wrong. Its stated confidence is systematically overconfident. It carries a little signal, but far too little to use as a quality metric.

The test for any proposed metric is one question: if this number moved five points, what would we do? If the honest answer is “nothing”, it is not a metric, it is a mood.

Leading and lagging

The number you care about is usually slow. Tickets resolved without a human, refunds issued in error, hours saved per week: these take a month to move and are contaminated by everything else happening in the business.

So you keep two. A lagging metric that is the actual point, measured monthly and treated as the truth. And a leading metric on your fixed set, measured in ten minutes, which you believe predicts the lagging one. The relationship between them is a claim, and you should check it at least once: when the offline number moved and the real one did not, you have learned something more valuable than either number.

Say what would change your mind

The last line of the file is the useful one. Finish the sentence “we would abandon this approach if…”. If nothing could produce that sentence, you are not running an evaluation. You are gathering support.

Most teams find this uncomfortable the first time, and most find that writing it makes the next three arguments shorter. The number is not there to prove you were right. It is there so that the room can stop guessing.