Getting Real Work out of a Model · Chapter 07

Fluency is not evidence

1:56 · Narrated with a synthetic voice. Captions are available in the player.

Read the transcript

A model does not sound unsure when it is unsure. Fluent, calm prose is its resting register, and it uses that register for a verified fact and for something it produced whole. Worse, fabrication can be more specific than the truth, because a citation is a highly patterned string, and one assembled from the most typical parts fits that pattern better than the messy real thing. So treat precision as a flag rather than a comfort. Three separate checks, in this order. Does the source exist. Does it say what is claimed. And does what it says support the inference. The first takes thirty seconds and catches the famous failure. The second and third are the ones that get into published work. Never verify by asking whether it is sure. That is a leading question aimed at a system trained towards agreement, and it will talk a model out of a correct answer as readily as a wrong one. Ask instead for the strongest argument that the answer is wrong, or offer two candidates without saying which you prefer. And remember the failure that leaves no mark. A summary of four payment terms that covers three reads exactly like one that covers four, so enumerate and count before you analyse. Then sort by consequence rather than by confidence. Check nothing when nothing is at stake. Spot-check what is recoverable. Verify independently anything expensive or irreversible.

Sources and further reading (1)