Confidence is the default setting
A model does not sound unsure when it is unsure. Fluent, calm, well-organised prose is its resting register, and it uses that register for a verified fact and for something it produced whole. You cannot read confidence off the tone, because the tone does not vary with the truth.
So you need other signals. Here are the ones that actually work.
Specificity is not evidence
This is the counterintuitive one, and it is the most useful thing in this lesson.
Under Section 14(3)(b) of the 2021 Act, filings are due within 21 days.
That looks like knowledge. Subsection, year, number. In a person, that precision would have been earned by reading the thing.
A fabricated answer can be exactly that specific, because the section number is generated the same way the sentence around it is. Invented citations, invented page numbers, invented case names, invented statistics with a decimal place — these are a well-documented failure mode, and they look more authoritative than the truth, which is often "it depends on your jurisdiction".
Precision tells you nothing about reliability. Treat a suspiciously exact figure as a flag, not a comfort.
Where guessing concentrates
Risk is not spread evenly. Be most careful with:
- Anything after the training cutoff. Prices, office holders, product versions, which company bought whom.
- Exact figures. Fares, tax thresholds, dosages, deadlines, fees. A Delhi metro fare or a Nairobi matatu fare quoted confidently may be four years stale.
- Citations and links. Both the existence and the content.
- Local law and local process. Rules that vary by country, state or city.
- Small or local organisations. A neighbourhood clinic's opening hours were never in the training data in any reliable form.
- Anything you are pushing hard for. If you ask three times for a source and there isn't one, you may get one anyway.
Four checks that take under a minute
1. Ask again in a fresh chat. Same question, new conversation. If you get a different answer, it is guessing. If you get the same answer, that is weak reassurance, not proof — a consistent error is still consistent.
2. Ask what would make it wrong. "Which parts of this are you least sure about, and why?" and "What would I need to check before relying on this?" Models are imperfect at self-assessment but not useless at it, and this often surfaces the shaky sentence. Do not read the answer as a probability.
3. Check the cheapest verifiable thing. Open the link. Search the quote. Confirm one number. If a citation turns out to be invented, distrust the whole passage around it, not just that line.
4. Give it a way out, in advance. "If you are not sure, say so." "If the document does not state this, write 'not stated'." "If you need a fact I have not given you, ask." Without an alternative to guessing, guessing is what the shape of the exchange calls for.
The asymmetry worth remembering
A model saying "I am confident" tells you almost nothing.
A model saying "I am not sure about this part" tells you a little more — it is not a reliable measurement, but volunteering doubt is at least a signal in the right direction, whereas confidence is the baseline.
So take hedges seriously and take assurances lightly. That is the opposite of how we read people, which is exactly why it takes practice.
The one habit that matters
Decide, before you ask, what you would do if this answer were wrong.
If the answer is "nothing much" — a first draft, a brainstorm, an explanation you will test against your own understanding — then move fast and do not worry.
If the answer is "I would send the wrong figure to a client", or "I would miss a filing deadline", or "I would take the wrong dose", then the model's job is to tell you where to look, and something else has to confirm it. Use it to generate the question, not to close it.
That single distinction will protect you better than any prompt.
Before you move on