{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"claims-about-a-document","title":"Checking a claim about a document you have","url":"https://addaly.com/learn/prompting/claims-about-a-document","position":58,"of":72,"minutes":8,"answer":"A claim about a document is checkable in seconds by searching a middle fragment of the quote and reading the whole sentence around it; and the sentence-level qualifiers are where summaries reverse meaning.","keyIdea":"A claim about a document is checkable in seconds by searching a middle fragment of the quote and reading the whole sentence around it; and the sentence-level qualifiers are where summaries reverse meaning."},"body":{"format":"markdown","text":"## The easiest verification there is, and nobody does it\n\nWhen the source is a document in front of you, checking is nearly free: a text search and ten seconds of reading. Yet this is where people check least, because an answer about a document you supplied feels safe. The model has the document. What could go wrong?\n\nFour things, and they have distinct signatures.\n\n## 1. The quote is not in the document\n\nThe simplest failure and the easiest catch. Search for a distinctive five-word fragment of the quote. If it is not there, nothing else in that answer is trustworthy either: treat one invented quote as evidence about the whole passage, not as an isolated slip.\n\nUse an exact phrase search, and search for a *middle* fragment rather than the opening, since openings get paraphrased more often than the body.\n\n## 2. The quote is real and the interpretation reverses it\n\nThe most dangerous, because the checkable part checks out.\n\nWatch for **negation** especially. \"The supplier shall not be liable except where negligence is established\" is easy to summarise as either \"not liable\" or \"liable when negligent\", and those are different contracts. Anything with *unless*, *except*, *provided that*, *save where*, *notwithstanding* deserves a second read of the original sentence, not the summary of it.\n\n**Scope** is the other one. A clause that applies only to Schedule 2 items, only during the initial term, only where written notice was given. Summaries lose qualifiers, because qualifiers are the least prominent part of a sentence and the first thing compression drops.\n\n## 3. The claim is true and incomplete\n\nThere are four payment terms and it found three. The answer is accurate about the three. Nothing signals the fourth. This is the omission problem, and it has its own lesson later in this module.\n\n## 4. It answered about the wrong document\n\nIf you pasted several (old policy and new, quote A and quote B, two versions of a draft) an answer can silently blend them or address the wrong one. This is why the labelling lesson exists, and why the answer should name which block each claim came from.\n\n## The prompt that makes all four checkable\n\n```\nFor each finding: quote the exact sentence, give its section or\nclause number, and name which document it came from.\nWhere a sentence contains a condition or exception (unless, except,\nprovided that, only where), quote the whole sentence including the\nqualifier, and state the condition separately in plain words.\nIf you cannot quote a sentence, do not make the claim.\n```\n\nThat is four lines and it addresses the first, second and fourth failures directly. The third needs a different move, which is asking for an enumeration before an analysis.\n\n## The check, in order of speed\n\n1. **Search a middle fragment of each quote.** Ten seconds. Catches invention.\n2. **Read the whole sentence around the quote**, not the quoted part. Catches truncation that removes a qualifier.\n3. **Read the sentence before and after.** Catches scope errors: the sentence that says \"the following applies only to renewals\".\n4. **Count.** If the answer lists three of something, ask whether there are exactly three.\n\nSteps one and two take under a minute for a five-finding answer and catch the great majority of what goes wrong.\n\n## Where this matters most\n\nContracts, policies, regulations, medical letters, tender documents, insurance terms, tenancy agreements. Anything where a condition or an exception carries the meaning, and where the summary sounds authoritative because it is fluent and confident about a document you have not read carefully yourself.\n\nThat last clause is the real risk. The reason people accept an incorrect summary is not that it was persuasive. It is that they were using it precisely because they had not read the document.\n\n## A worked example\n\nA clause reads: *\"The Licensor shall not unreasonably withhold consent to assignment, save where the proposed assignee is a competitor of the Licensor or where any sum remains outstanding.\"*\n\nA summary said: consent to assignment cannot be unreasonably withheld.\n\nEvery word of that is in the clause. It is also, for a company with an unpaid invoice, exactly wrong: the second exception applies and consent can be withheld outright. The quote checked out, the section number was right, and the answer was still useless in the only situation the reader was in.\n\nThis is why \"quote the whole sentence including the qualifier\" is in the prompt above, and why reading the original sentence beats reading the summary of it."},"check":{"question":"A tenant asks whether her lease allows the landlord to enter without notice. The answer quotes a real clause accurately and concludes he cannot. What single check is most likely to reveal an error, if there is one?","options":[{"text":"Searching the lease for the word \"notice\" to see how many times it appears.","correct":false,"tempting":"A reasonable sweep for omissions, but it does not test the interpretation of the clause that was quoted."},{"text":"Asking the model to double-check its conclusion.","correct":false,"tempting":"A leading question to a system that leans towards agreement; it will confirm."},{"text":"Reading the full sentence and the sentences either side of the quoted clause, looking for an exception or a scope limit.","correct":true},{"text":"Asking a second model the same question and comparing.","correct":false,"tempting":"Useful in general, and two models reading the same clause can share the same misreading of a qualifier."}]},"prev":{"slug":"search-grounded-answers","title":"When it can search, and what that fixes","url":"https://addaly.com/learn/prompting/search-grounded-answers"},"next":{"slug":"checking-numbers","title":"Checking a number in under a minute","url":"https://addaly.com/learn/prompting/checking-numbers"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Checking a claim about a document you have', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/claims-about-a-document (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Checking a claim about a document you have","courseTitle":"Getting Real Work out of a Model","module":{"index":7,"slug":"checking-the-answer","title":"Checking the answer"},"answer":null,"keyIdea":"A claim about a document is checkable in seconds by searching a middle fragment of the quote and reading the whole sentence around it; and the sentence-level qualifiers are where summaries reverse meaning.","hero":{"kind":"plate","image":"https://addaly.com/art/modules/prompting/checking-the-answer.webp","alt":"A clinic reception desk with a rubber stamp, a ruled sheet, spectacles, an open book and a telephone.","caption":"Verification is a desk habit rather than a feeling: the claim, the register it is checked against, and the stamp that only goes on afterwards."},"blocks":[{"t":"h2","id":"the-easiest-verification-there-is-and-nobody-does-it","runs":[{"text":"The easiest verification there is, and nobody does it"}]},{"t":"p","runs":[{"text":"When the source is a document in front of you, checking is nearly free: a text search and ten seconds of reading. Yet this is where people check least, because an answer about a document you supplied feels safe. The model has the document. What could go wrong?"}]},{"t":"p","runs":[{"text":"Four things, and they have distinct signatures."}]},{"t":"h2","id":"1-the-quote-is-not-in-the-document","runs":[{"text":"1. The quote is not in the document"}]},{"t":"p","runs":[{"text":"The simplest failure and the easiest catch. Search for a distinctive five-word fragment of the quote. If it is not there, nothing else in that answer is trustworthy either: treat one invented quote as evidence about the whole passage, not as an isolated slip."}]},{"t":"p","runs":[{"text":"Use an exact phrase search, and search for a "},{"text":"middle","i":true},{"text":" fragment rather than the opening, since openings get paraphrased more often than the body."}]},{"t":"h2","id":"2-the-quote-is-real-and-the-interpretation-reverses-it","runs":[{"text":"2. The quote is real and the interpretation reverses it"}]},{"t":"p","runs":[{"text":"The most dangerous, because the checkable part checks out."}]},{"t":"p","runs":[{"text":"Watch for "},{"text":"negation","b":true},{"text":" especially. “The supplier shall not be liable except where negligence is established” is easy to summarise as either “not liable” or “liable when negligent”, and those are different contracts. Anything with "},{"text":"unless","i":true},{"text":", "},{"text":"except","i":true},{"text":", "},{"text":"provided that","i":true},{"text":", "},{"text":"save where","i":true},{"text":", "},{"text":"notwithstanding","i":true},{"text":" deserves a second read of the original sentence, not the summary of it."}]},{"t":"p","runs":[{"text":"Scope","b":true},{"text":" is the other one. A clause that applies only to Schedule 2 items, only during the initial term, only where written notice was given. Summaries lose qualifiers, because qualifiers are the least prominent part of a sentence and the first thing compression drops."}]},{"t":"h2","id":"3-the-claim-is-true-and-incomplete","runs":[{"text":"3. The claim is true and incomplete"}]},{"t":"p","runs":[{"text":"There are four payment terms and it found three. The answer is accurate about the three. Nothing signals the fourth. This is the omission problem, and it has its own lesson later in this module."}]},{"t":"h2","id":"4-it-answered-about-the-wrong-document","runs":[{"text":"4. It answered about the wrong document"}]},{"t":"p","runs":[{"text":"If you pasted several (old policy and new, quote A and quote B, two versions of a draft) an answer can silently blend them or address the wrong one. This is why the labelling lesson exists, and why the answer should name which block each claim came from."}]},{"t":"h2","id":"the-prompt-that-makes-all-four-checkable","runs":[{"text":"The prompt that makes all four checkable"}]},{"t":"code","lang":"","label":"Text","text":"For each finding: quote the exact sentence, give its section or\nclause number, and name which document it came from.\nWhere a sentence contains a condition or exception (unless, except,\nprovided that, only where), quote the whole sentence including the\nqualifier, and state the condition separately in plain words.\nIf you cannot quote a sentence, do not make the claim."},{"t":"p","runs":[{"text":"That is four lines and it addresses the first, second and fourth failures directly. The third needs a different move, which is asking for an enumeration before an analysis."}]},{"t":"h2","id":"the-check-in-order-of-speed","runs":[{"text":"The check, in order of speed"}]},{"t":"ol","start":1,"items":[[{"text":"Search a middle fragment of each quote.","b":true},{"text":" Ten seconds. Catches invention."}],[{"text":"Read the whole sentence around the quote","b":true},{"text":", not the quoted part. Catches truncation that removes a qualifier."}],[{"text":"Read the sentence before and after.","b":true},{"text":" Catches scope errors: the sentence that says “the following applies only to renewals”."}],[{"text":"Count.","b":true},{"text":" If the answer lists three of something, ask whether there are exactly three."}]]},{"t":"p","runs":[{"text":"Steps one and two take under a minute for a five-finding answer and catch the great majority of what goes wrong."}]},{"t":"h2","id":"where-this-matters-most","runs":[{"text":"Where this matters most"}]},{"t":"p","runs":[{"text":"Contracts, policies, regulations, medical letters, tender documents, insurance terms, tenancy agreements. Anything where a condition or an exception carries the meaning, and where the summary sounds authoritative because it is fluent and confident about a document you have not read carefully yourself."}]},{"t":"p","runs":[{"text":"That last clause is the real risk. The reason people accept an incorrect summary is not that it was persuasive. It is that they were using it precisely because they had not read the document."}]},{"t":"h2","id":"a-worked-example","runs":[{"text":"A worked example"}]},{"t":"p","runs":[{"text":"A clause reads: "},{"text":"”The Licensor shall not unreasonably withhold consent to assignment, save where the proposed assignee is a competitor of the Licensor or where any sum remains outstanding.”","i":true}]},{"t":"p","runs":[{"text":"A summary said: consent to assignment cannot be unreasonably withheld."}]},{"t":"p","runs":[{"text":"Every word of that is in the clause. It is also, for a company with an unpaid invoice, exactly wrong: the second exception applies and consent can be withheld outright. The quote checked out, the section number was right, and the answer was still useless in the only situation the reader was in."}]},{"t":"p","runs":[{"text":"This is why “quote the whole sentence including the qualifier” is in the prompt above, and why reading the original sentence beats reading the summary of it."}]}],"check":{"question":[{"text":"A tenant asks whether her lease allows the landlord to enter without notice. The answer quotes a real clause accurately and concludes he cannot. What single check is most likely to reveal an error, if there is one?"}],"options":[{"text":[{"text":"Searching the lease for the word “notice” to see how many times it appears."}],"correct":false,"tempting":[{"text":"A reasonable sweep for omissions, but it does not test the interpretation of the clause that was quoted."}]},{"text":[{"text":"Asking the model to double-check its conclusion."}],"correct":false,"tempting":[{"text":"A leading question to a system that leans towards agreement; it will confirm."}]},{"text":[{"text":"Reading the full sentence and the sentences either side of the quoted clause, looking for an exception or a scope limit."}],"correct":true,"tempting":null},{"text":[{"text":"Asking a second model the same question and comparing."}],"correct":false,"tempting":[{"text":"Useful in general, and two models reading the same clause can share the same misreading of a qualifier."}]}],"order":[1,0,3,2],"why":null},"prev":{"slug":"search-grounded-answers","title":"When it can search, and what that fixes","minutes":8},"next":{"slug":"checking-numbers","title":"Checking a number in under a minute","minutes":8}}}