{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"when-the-answer-is-not-in-the-file","title":"Checking a grounded answer in under a minute","url":"https://addaly.com/learn/ai-for-small-business/when-the-answer-is-not-in-the-file","position":28,"of":88,"minutes":8,"answer":"Check a grounded answer by confirming a citation exists, that the quoted text is really in the source, that it supports the claim rather than merely sitting near it, and that the source is current.","keyIdea":"Check a grounded answer by confirming a citation exists, that the quoted text is really in the source, that it supports the claim rather than merely sitting near it, and that the source is current, and treat a completely confident answer with no gaps as the one most worth examining."},"body":{"format":"markdown","text":"## The check has to be mechanical\n\nYou have grounded the system, the answers carry citations, and the whole thing is working. The remaining question is how you satisfy yourself, quickly, that any particular answer is real.\n\nReading is not enough. A fabricated answer and a correct one read identically, which has been the point of this course from the beginning. What you need is a procedure short enough that you will actually do it every time.\n\n## The sixty-second check\n\n**1. Is there a citation at all?** No citation means the sentence is unsupported, whatever it says. Some tools cite only some claims; treat an uncited sentence as memory.\n\n**2. Does the quoted text exist?** Search the source document for a distinctive phrase from the quote. This is the step that catches fabricated citations, and it takes about ten seconds with a find box.\n\n**3. Does the quote support the claim?** The commonest real-world failure is not an invented quote. It is a genuine quote that says something adjacent. The passage is about deposits; the answer is about final payments. Both are real, and the answer is wrong.\n\n**4. Is the source current?** Look at the date in the document. A perfect quote from a superseded version is a wrong answer with excellent provenance.\n\n**5. Would a missing document change this?** Especially for any answer of the form \"there is no...\". Absence findings do not survive this step, ever.\n\nFive checks, under a minute for a short answer, and mechanical enough to hand to somebody else.\n\n> Figure: The sixty-second check on a grounded answer. Steps: 1. Is there a citation (No citation means the sentence is unsupported, whatever it says. Treat an uncited sentence as memory.); 2. Does the quote exist (Search the source for a distinctive phrase from it. Ten seconds with a find box, and it catches a fabricated citation.); 3. Does it support the claim (The commonest failure is a real quote that says something adjacent. The passage is about deposits; the answer is about final payments.); 4. Is the source current (A perfect quote from a superseded version is a wrong answer with excellent provenance.); 5. Would a missing file change it (Any answer of the form there is no such clause does not survive this step, ever.). Mechanical enough to write on a card beside the screen and hand to somebody else. The polished, entirely confident answer with no gaps is the one to look at hardest, because real documents have holes in them.\n\n## Making the system easier to check\n\nAsk for the format that makes checking fast, and it costs you nothing:\n\n```text\nAnswer using only the sources. After each sentence, put\n[document name, section] and the exact quoted text it\ncame from. If a sentence is not supported by a source,\nmark it [UNSUPPORTED] and keep it separate at the end.\n```\n\nThat last clause is worth more than it looks. Rather than forbidding unsupported statements (which usually just makes them disappear into the supported ones) you give them somewhere to go. Useful context ends up in a clearly-labelled section you can read differently, which is a far better outcome than a confident refusal or a blended answer where you cannot tell the parts apart.\n\n## When to check everything, and when to sample\n\n**Check every answer** when it goes to a customer, when money or a commitment is involved, when the question is unusual, or during the first fortnight with any new setup.\n\n**Sample** for internal, routine, low-stakes use once the system has a track record. Ten answers a week, checked properly, is enough to notice a change.\n\nThe thing to avoid is the middle: a vague intention to check when something looks odd. Nothing looks odd. That is the whole problem.\n\n## What a good answer looks like\n\nIt is worth knowing what you are aiming at, because it is not the most impressive-looking output.\n\nA good grounded answer is often slightly awkward. It says \"the terms document, section 4, states a fourteen-day return window\" instead of a smooth paragraph about your excellent returns policy. It says NOT IN SOURCES for the part you did not document. It carries quotes that turn out, when you look, to say exactly what it claimed.\n\nThe polished, comprehensive, entirely confident answer with no gaps and no hedging is the one to look at hardest. Real documents have holes in them, and a system that never reports a hole is not reporting.\n\n## Teach the check, not the tool\n\nIf somebody else will use the system, the five checks are what you hand over, not a demonstration of how good the answers are.\n\nWrite them on one card next to the screen. Show a real failure: an answer whose quote turns out to be about the adjacent policy. People generalise from one concrete failure far better than from a warning, and it takes two minutes.\n\nThen agree what happens when a check fails. Usually: do not send it, fix the document if the document was the problem, and tell whoever owns the library. Without that last step, the same failure is caught fifteen separate times by fifteen different people and fixed by none of them.\n\n## When the checking never gets cheaper\n\nIf, after a month, the five checks still take real effort on every answer, that is information. It means the questions being asked are not ones your documents answer, and the honest response is either to write the missing document or to stop using the system for that class of question.\n\nA grounded assistant should get easier to supervise as your library matures, because the answers converge on material you have written deliberately. If supervision is not getting cheaper, the library is the problem, not your diligence."},"check":{"question":"An assistant returns an answer with a citation. The quoted passage is genuinely in the handbook, word for word. Which of the five checks is still most likely to fail?","options":[{"text":"Whether the citation exists, since some tools attach a source label to sentences generated from memory.","correct":false,"tempting":"Already satisfied by the premise; a citation is present and the quote has been confirmed to be genuine, so this check has passed."},{"text":"Whether the quote actually supports the claim, since a real passage about an adjacent subject is the commonest failure once fabrication is ruled out.","correct":true},{"text":"Whether the quoted text exists, since verbatim matching can be defeated by small changes in whitespace or punctuation between the answer and the source document.","correct":false,"tempting":"Raises a matching technicality when the premise states the passage is present word for word, so this check has already been performed and passed."},{"text":"Whether the source is current, since a genuine quote from a superseded document produces a wrong answer with good provenance.","correct":false,"tempting":"A real and important failure, but it depends on the business having a superseded copy in the library, whereas the adjacent-passage problem occurs with a single perfectly current document."}]},"prev":{"slug":"keeping-the-knowledge-current","title":"The stale document problem","url":"https://addaly.com/learn/ai-for-small-business/keeping-the-knowledge-current"},"next":{"slug":"customer-messages","title":"Answering Customers Without Losing Your Voice","url":"https://addaly.com/learn/ai-for-small-business/customer-messages"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Checking a grounded answer in under a minute', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/when-the-answer-is-not-in-the-file (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Checking a grounded answer in under a minute","courseTitle":"AI for a Small Business","module":{"index":3,"slug":"your-own-knowledge","title":"Your own documents, and getting straight answers out of them"},"answer":null,"keyIdea":"Check a grounded answer by confirming a citation exists, that the quoted text is really in the source, that it supports the claim rather than merely sitting near it, and that the source is current, and treat a completely confident answer with no gaps as the one most worth examining.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"Is there a citation","detail":"No citation means the sentence is unsupported, whatever it says. Treat an uncited sentence as memory."},{"label":"Does the quote exist","detail":"Search the source for a distinctive phrase from it. Ten seconds with a find box, and it catches a fabricated citation."},{"label":"Does it support the claim","detail":"The commonest failure is a real quote that says something adjacent. The passage is about deposits; the answer is about final payments."},{"label":"Is the source current","detail":"A perfect quote from a superseded version is a wrong answer with excellent provenance."},{"label":"Would a missing file change it","detail":"Any answer of the form there is no such clause does not survive this step, ever."}],"title":"The sixty-second check on a grounded answer","caption":"Mechanical enough to write on a card beside the screen and hand to somebody else. The polished, entirely confident answer with no gaps is the one to look at hardest, because real documents have holes in them."},"alt":"Figure: The sixty-second check on a grounded answer. Steps: 1. Is there a citation (No citation means the sentence is unsupported, whatever it says. Treat an uncited sentence as memory.); 2. Does the quote exist (Search the source for a distinctive phrase from it. Ten seconds with a find box, and it catches a fabricated citation.); 3. Does it support the claim (The commonest failure is a real quote that says something adjacent. The passage is about deposits; the answer is about final payments.); 4. Is the source current (A perfect quote from a superseded version is a wrong answer with excellent provenance.); 5. Would a missing file change it (Any answer of the form there is no such clause does not survive this step, ever.). Mechanical enough to write on a card beside the screen and hand to somebody else. The polished, entirely confident answer with no gaps is the one to look at hardest, because real documents have holes in them."}},"blocks":[{"t":"h2","id":"the-check-has-to-be-mechanical","runs":[{"text":"The check has to be mechanical"}]},{"t":"p","runs":[{"text":"You have grounded the system, the answers carry citations, and the whole thing is working. The remaining question is how you satisfy yourself, quickly, that any particular answer is real."}]},{"t":"p","runs":[{"text":"Reading is not enough. A fabricated answer and a correct one read identically, which has been the point of this course from the beginning. What you need is a procedure short enough that you will actually do it every time."}]},{"t":"h2","id":"the-sixty-second-check","runs":[{"text":"The sixty-second check"}]},{"t":"p","runs":[{"text":"1. Is there a citation at all?","b":true},{"text":" No citation means the sentence is unsupported, whatever it says. Some tools cite only some claims; treat an uncited sentence as memory."}]},{"t":"p","runs":[{"text":"2. Does the quoted text exist?","b":true},{"text":" Search the source document for a distinctive phrase from the quote. This is the step that catches fabricated citations, and it takes about ten seconds with a find box."}]},{"t":"p","runs":[{"text":"3. Does the quote support the claim?","b":true},{"text":" The commonest real-world failure is not an invented quote. It is a genuine quote that says something adjacent. The passage is about deposits; the answer is about final payments. Both are real, and the answer is wrong."}]},{"t":"p","runs":[{"text":"4. Is the source current?","b":true},{"text":" Look at the date in the document. A perfect quote from a superseded version is a wrong answer with excellent provenance."}]},{"t":"p","runs":[{"text":"5. Would a missing document change this?","b":true},{"text":" Especially for any answer of the form “there is no...”. Absence findings do not survive this step, ever."}]},{"t":"p","runs":[{"text":"Five checks, under a minute for a short answer, and mechanical enough to hand to somebody else."}]},{"t":"h2","id":"making-the-system-easier-to-check","runs":[{"text":"Making the system easier to check"}]},{"t":"p","runs":[{"text":"Ask for the format that makes checking fast, and it costs you nothing:"}]},{"t":"code","lang":"text","label":"Text","text":"Answer using only the sources. After each sentence, put\n[document name, section] and the exact quoted text it\ncame from. If a sentence is not supported by a source,\nmark it [UNSUPPORTED] and keep it separate at the end."},{"t":"p","runs":[{"text":"That last clause is worth more than it looks. Rather than forbidding unsupported statements (which usually just makes them disappear into the supported ones) you give them somewhere to go. Useful context ends up in a clearly-labelled section you can read differently, which is a far better outcome than a confident refusal or a blended answer where you cannot tell the parts apart."}]},{"t":"h2","id":"when-to-check-everything-and-when-to-sample","runs":[{"text":"When to check everything, and when to sample"}]},{"t":"p","runs":[{"text":"Check every answer","b":true},{"text":" when it goes to a customer, when money or a commitment is involved, when the question is unusual, or during the first fortnight with any new setup."}]},{"t":"p","runs":[{"text":"Sample","b":true},{"text":" for internal, routine, low-stakes use once the system has a track record. Ten answers a week, checked properly, is enough to notice a change."}]},{"t":"p","runs":[{"text":"The thing to avoid is the middle: a vague intention to check when something looks odd. Nothing looks odd. That is the whole problem."}]},{"t":"h2","id":"what-a-good-answer-looks-like","runs":[{"text":"What a good answer looks like"}]},{"t":"p","runs":[{"text":"It is worth knowing what you are aiming at, because it is not the most impressive-looking output."}]},{"t":"p","runs":[{"text":"A good grounded answer is often slightly awkward. It says “the terms document, section 4, states a fourteen-day return window” instead of a smooth paragraph about your excellent returns policy. It says NOT IN SOURCES for the part you did not document. It carries quotes that turn out, when you look, to say exactly what it claimed."}]},{"t":"p","runs":[{"text":"The polished, comprehensive, entirely confident answer with no gaps and no hedging is the one to look at hardest. Real documents have holes in them, and a system that never reports a hole is not reporting."}]},{"t":"h2","id":"teach-the-check-not-the-tool","runs":[{"text":"Teach the check, not the tool"}]},{"t":"p","runs":[{"text":"If somebody else will use the system, the five checks are what you hand over, not a demonstration of how good the answers are."}]},{"t":"p","runs":[{"text":"Write them on one card next to the screen. Show a real failure: an answer whose quote turns out to be about the adjacent policy. People generalise from one concrete failure far better than from a warning, and it takes two minutes."}]},{"t":"p","runs":[{"text":"Then agree what happens when a check fails. Usually: do not send it, fix the document if the document was the problem, and tell whoever owns the library. Without that last step, the same failure is caught fifteen separate times by fifteen different people and fixed by none of them."}]},{"t":"h2","id":"when-the-checking-never-gets-cheaper","runs":[{"text":"When the checking never gets cheaper"}]},{"t":"p","runs":[{"text":"If, after a month, the five checks still take real effort on every answer, that is information. It means the questions being asked are not ones your documents answer, and the honest response is either to write the missing document or to stop using the system for that class of question."}]},{"t":"p","runs":[{"text":"A grounded assistant should get easier to supervise as your library matures, because the answers converge on material you have written deliberately. If supervision is not getting cheaper, the library is the problem, not your diligence."}]}],"check":{"question":[{"text":"An assistant returns an answer with a citation. The quoted passage is genuinely in the handbook, word for word. Which of the five checks is still most likely to fail?"}],"options":[{"text":[{"text":"Whether the citation exists, since some tools attach a source label to sentences generated from memory."}],"correct":false,"tempting":[{"text":"Already satisfied by the premise; a citation is present and the quote has been confirmed to be genuine, so this check has passed."}]},{"text":[{"text":"Whether the quote actually supports the claim, since a real passage about an adjacent subject is the commonest failure once fabrication is ruled out."}],"correct":true,"tempting":null},{"text":[{"text":"Whether the quoted text exists, since verbatim matching can be defeated by small changes in whitespace or punctuation between the answer and the source document."}],"correct":false,"tempting":[{"text":"Raises a matching technicality when the premise states the passage is present word for word, so this check has already been performed and passed."}]},{"text":[{"text":"Whether the source is current, since a genuine quote from a superseded document produces a wrong answer with good provenance."}],"correct":false,"tempting":[{"text":"A real and important failure, but it depends on the business having a superseded copy in the library, whereas the adjacent-passage problem occurs with a single perfectly current document."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"keeping-the-knowledge-current","title":"The stale document problem","minutes":8},"next":{"slug":"customer-messages","title":"Answering Customers Without Losing Your Voice","minutes":8}}}