This chapter’s goalSort a failure into design, capability or misuse, read a benchmark or a demo for what it leaves out, and set how much checking a use needs by what it costs if wrong.
The first direct image and Webb’s own first direct image are different claims. Check the telescope, object and date before repeating either one.
Figure
A little at a time
Explore an idea, try it, then check what stays.
Step 1 of 7Free · no account needed
Lesson steps
The answer sounds certain. Where is the evidence?
On 6 February 2023, Google announced its experimental Bard service and opened it to trusted testers. An early example asked for discoveries from the James Webb Space Telescope to share with a nine-year-old. The third bullet in the answer attributed the first pictures of a planet outside our solar system to Webb. The European Southern Observatory reports that the earlier direct image of 2M1207 b was obtained in 2004 with the Very Large Telescope. Follow-up observations supported the identification in 2005. In 2022, NASA reported Webb’s own first direct image of an exoplanet. The word “own” changes what the claim means.
The answer is an example of a wrong factual claim. The example does not tell us which part of Bard’s system produced it.
Quick check
Before the answer: how far was Bard’s sentence from a true one?
Next pieces, not a list of verified facts
Many text-generating language models are first trained to predict a next token from previous tokens. Tokens are pieces of text, which may be whole words, parts of words or punctuation. The model computes scores over possible next tokens. A decoding rule chooses one and the process continues. Some systems add other training objectives or steps; this is a useful sketch of a common text generation loop.
Prediction can learn useful patterns about grammar, language and the world. It does not by itself look up an original source or prove that a statement is true. A deployed assistant may also retrieve documents, use tools or run checks. The reliability of those checks depends on whether they ran, what evidence they found and how the assistant used it.
This sketches text generation itself. A deployed assistant may also retrieve sources or use tools. Check whether that happened and whether the evidence supports the answer.
Where the assistant’s behaviour comes from
Pretraining teaches patterns from large amounts of data. The exact training mix and objectives depend on the model.
Instruction tuning can teach how to respond to requests by learning from examples. Preference training can steer the assistant towards responses people prefer; the InstructGPT paper describes one method. In a separate set of experiments, Sharma and colleagues found that some assistants sometimes agreed with a user’s stated view even when it was wrong. That does not mean every assistant will do so in every setting.
Models can also show signals related to their uncertainty. Kadavath and colleagues tested ways to elicit such signals. A confident style in ordinary text is not one of those verified measurements. Treat confidence as something you test under known conditions.
Try it: open the source before using the fact
Write down one factual sentence from a generated explanation. Underline its date, named object and action. Find an independent source that could verify those details. Mark each item “supported,” “contradicted” or “not addressed.” If the source leaves a gap, keep the gap visible. A citation at the end of a paragraph is useful only when it supports the particular claim you are using.
Check your understanding
The same assistant repeats a disputed claim and gives a confident reason. What would add independent evidence?
Pick the one you would defend. Your answer stays private.