{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"why-retrieval-beats-memory","title":"Grounding: answering from your material rather than its memory","url":"https://addaly.com/learn/ai-for-small-business/why-retrieval-beats-memory","position":21,"of":88,"minutes":9,"answer":"Grounding converts the task from recall, where models fabricate, to reading comprehension, where they are strong, but it inherits everything wrong with your documents.","keyIdea":"Grounding converts the task from recall, where models fabricate, to reading comprehension, where they are strong, but it inherits everything wrong with your documents, so a grounded answer from a stale price list is a confident error with a citation attached."},"body":{"format":"markdown","text":"## Two ways to get an answer\n\nAsk a model what your refund policy is and it will tell you something. Where that something came from is the whole question.\n\n**From memory.** The model produces the most likely words following your question, drawn from everything it read during training. Fast, free, and unconnected to your business.\n\n**From a source.** The tool finds relevant text (your policy document, your price list, your past emails) puts it in front of the model, and asks it to answer using that. Slower, sometimes costs a little more, and the answer is about your business.\n\nThe second is called **grounding**, or retrieval-augmented generation when people want it to sound technical. It is the single most useful architectural idea for a small business, and it explains why a tool that has your documents is a different class of thing from one that does not.\n\n## Why it works\n\nIt changes the task from recall to reading comprehension.\n\nRecall is where models are weakest: they are reconstructing, not retrieving, so a plausible answer and a true one are produced by the same process. Reading comprehension is where they are strongest. Given a passage that contains the answer, extracting and rephrasing it is close to the thing they do best.\n\nThat is the whole mechanism, and it explains a pattern you will notice: a model that invents wildly about your business becomes remarkably reliable the moment you paste your actual price list. Nothing about the model changed. The task changed.\n\n## What you get beyond accuracy\n\nThree things, and the second is the one that matters most in practice.\n\n**Currency.** Change the document, and the answer changes. No retraining, no waiting for a new model. Your price rise takes effect the moment you edit the file.\n\n**Citations.** A grounded system can say which document and which line an answer came from. This converts checking from re-reading everything to glancing at one quoted line. It is the difference between an answer you have to trust and an answer you can verify.\n\n**Confinement.** With the book closed, questions outside your material get a clean refusal rather than an improvised answer. For anything customer-facing this is the property you actually want.\n\n## The failures grounding does not fix\n\nBeing honest about this matters, because \"we use RAG\" gets said as though it settles the question.\n\n**Bad sources produce bad answers, confidently.** A grounded system quoting last year's price list is worse than an ungrounded one, because it now has a citation. Everything depends on the documents being right and current.\n\n**Retrieval can miss.** If the relevant passage is not fetched, the model answers from what it did fetch, and may fall back on memory anyway unless firmly instructed not to.\n\n**It can still fabricate a citation.** Less often, but it happens: a quoted line that reads perfectly and appears nowhere in the source. Which is why the check is against the document, not against the presence of a quote.\n\n**It does not add judgement.** The system can tell you your policy says fourteen days. It cannot tell you whether to make an exception for this customer.\n\n## Doing this without building anything\n\nYou do not need a developer, and most small businesses should not have one for this.\n\n- **Paste the relevant page** into the chat with your question. Crude, free, and it is real grounding.\n- **Standing instructions** holding your facts sheet. Grounding on your most-used material, permanently.\n- **A document tool**: a project with files attached, or a purpose-built notebook product. Grounding across a small library.\n- **A local setup** if the documents are sensitive: open tools such as AnythingLLM or Open WebUI paired with Ollama do this on your own machine, free.\n\nThe instruction that turns any of them into a grounded system is the same one from module two: *answer only from the sources below; if it is not there, say NOT IN SOURCES.* Without that line, all four arrangements will quietly fall back on memory when retrieval comes up short, which is precisely when you least want them to.\n\n## Keeping the sources small\n\nA common instinct is to throw everything at it (every policy, every past email, ten years of quotes) on the grounds that more material means better answers.\n\nIt does not. A bigger library means retrieval has more chances to fetch something that merely resembles the answer, and it means more opportunities for a stale document to win. Businesses that get good results from grounding usually have a small, curated set: the current price list, the current terms, the current procedures, and nothing superseded.\n\nDelete or archive the old versions rather than leaving them in the folder marked \"old\". A file named `prices-2024-OLD.pdf` is, to a retrieval system, just a document about prices.\n\n## A worked example\n\nA guesthouse put six documents into a notebook tool: rates, house rules, directions, a cancellation policy, a local-recommendations page and an FAQ built from real emails. Total, about nine pages.\n\nAnswers to guest questions became reliable enough to draft from, and the owner could see which page each answer came from. The whole set-up took an afternoon, cost nothing beyond a free tier, and the maintenance is editing a document when something changes, which she was doing anyway."},"check":{"question":"A business grounds its assistant on a folder of policy documents. Answers become far more accurate, but one reply confidently states a 30-day return window that no document contains. What is the most useful conclusion?","options":[{"text":"Grounding has failed and the folder should be rebuilt, since a single fabrication indicates the retrieval index is corrupt.","correct":false,"tempting":"Escalates one failure into an infrastructure problem; retrieval that answers most questions correctly is working, and rebuilding it addresses nothing about this reply."},{"text":"The 30-day figure is probably an industry norm supplied from the model's memory, so the source folder should be re-checked to see whether returns are covered anywhere in the documents at all.","correct":false,"tempting":"Identifies the likely origin and stops one step short: the immediate lesson is about the missing instruction, not only about a gap in the documents."},{"text":"Grounding is unsuitable for policy questions because policies contain exceptions that retrieval cannot represent.","correct":false,"tempting":"Rejects the technique over a general worry about exceptions, when the reported problem is a fabricated number rather than a mishandled edge case."},{"text":"Retrieval found nothing relevant and the model fell back on memory, which means the answer-only-from-sources instruction is missing or too weak.","correct":true}]},"prev":{"slug":"asking-your-own-documents","title":"What actually happens when you upload a file","url":"https://addaly.com/learn/ai-for-small-business/asking-your-own-documents"},"next":{"slug":"embeddings-for-non-programmers","title":"Why it finds \"refund\" when the customer typed \"money back\"","url":"https://addaly.com/learn/ai-for-small-business/embeddings-for-non-programmers"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Grounding: answering from your material rather than its memory', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/why-retrieval-beats-memory (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Grounding: answering from your material rather than its memory","courseTitle":"AI for a Small Business","module":{"index":3,"slug":"your-own-knowledge","title":"Your own documents, and getting straight answers out of them"},"answer":null,"keyIdea":"Grounding converts the task from recall, where models fabricate, to reading comprehension, where they are strong, but it inherits everything wrong with your documents, so a grounded answer from a stale price list is a confident error with a citation attached.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-ways-to-get-an-answer","runs":[{"text":"Two ways to get an answer"}]},{"t":"p","runs":[{"text":"Ask a model what your refund policy is and it will tell you something. Where that something came from is the whole question."}]},{"t":"p","runs":[{"text":"From memory.","b":true},{"text":" The model produces the most likely words following your question, drawn from everything it read during training. Fast, free, and unconnected to your business."}]},{"t":"p","runs":[{"text":"From a source.","b":true},{"text":" The tool finds relevant text (your policy document, your price list, your past emails) puts it in front of the model, and asks it to answer using that. Slower, sometimes costs a little more, and the answer is about your business."}]},{"t":"p","runs":[{"text":"The second is called "},{"text":"grounding","b":true},{"text":", or retrieval-augmented generation when people want it to sound technical. It is the single most useful architectural idea for a small business, and it explains why a tool that has your documents is a different class of thing from one that does not."}]},{"t":"h2","id":"why-it-works","runs":[{"text":"Why it works"}]},{"t":"p","runs":[{"text":"It changes the task from recall to reading comprehension."}]},{"t":"p","runs":[{"text":"Recall is where models are weakest: they are reconstructing, not retrieving, so a plausible answer and a true one are produced by the same process. Reading comprehension is where they are strongest. Given a passage that contains the answer, extracting and rephrasing it is close to the thing they do best."}]},{"t":"p","runs":[{"text":"That is the whole mechanism, and it explains a pattern you will notice: a model that invents wildly about your business becomes remarkably reliable the moment you paste your actual price list. Nothing about the model changed. The task changed."}]},{"t":"h2","id":"what-you-get-beyond-accuracy","runs":[{"text":"What you get beyond accuracy"}]},{"t":"p","runs":[{"text":"Three things, and the second is the one that matters most in practice."}]},{"t":"p","runs":[{"text":"Currency.","b":true},{"text":" Change the document, and the answer changes. No retraining, no waiting for a new model. Your price rise takes effect the moment you edit the file."}]},{"t":"p","runs":[{"text":"Citations.","b":true},{"text":" A grounded system can say which document and which line an answer came from. This converts checking from re-reading everything to glancing at one quoted line. It is the difference between an answer you have to trust and an answer you can verify."}]},{"t":"p","runs":[{"text":"Confinement.","b":true},{"text":" With the book closed, questions outside your material get a clean refusal rather than an improvised answer. For anything customer-facing this is the property you actually want."}]},{"t":"h2","id":"the-failures-grounding-does-not-fix","runs":[{"text":"The failures grounding does not fix"}]},{"t":"p","runs":[{"text":"Being honest about this matters, because “we use RAG” gets said as though it settles the question."}]},{"t":"p","runs":[{"text":"Bad sources produce bad answers, confidently.","b":true},{"text":" A grounded system quoting last year’s price list is worse than an ungrounded one, because it now has a citation. Everything depends on the documents being right and current."}]},{"t":"p","runs":[{"text":"Retrieval can miss.","b":true},{"text":" If the relevant passage is not fetched, the model answers from what it did fetch, and may fall back on memory anyway unless firmly instructed not to."}]},{"t":"p","runs":[{"text":"It can still fabricate a citation.","b":true},{"text":" Less often, but it happens: a quoted line that reads perfectly and appears nowhere in the source. Which is why the check is against the document, not against the presence of a quote."}]},{"t":"p","runs":[{"text":"It does not add judgement.","b":true},{"text":" The system can tell you your policy says fourteen days. It cannot tell you whether to make an exception for this customer."}]},{"t":"h2","id":"doing-this-without-building-anything","runs":[{"text":"Doing this without building anything"}]},{"t":"p","runs":[{"text":"You do not need a developer, and most small businesses should not have one for this."}]},{"t":"ul","items":[[{"text":"Paste the relevant page","b":true},{"text":" into the chat with your question. Crude, free, and it is real grounding."}],[{"text":"Standing instructions","b":true},{"text":" holding your facts sheet. Grounding on your most-used material, permanently."}],[{"text":"A document tool","b":true},{"text":": a project with files attached, or a purpose-built notebook product. Grounding across a small library."}],[{"text":"A local setup","b":true},{"text":" if the documents are sensitive: open tools such as AnythingLLM or Open WebUI paired with Ollama do this on your own machine, free."}]]},{"t":"p","runs":[{"text":"The instruction that turns any of them into a grounded system is the same one from module two: "},{"text":"answer only from the sources below; if it is not there, say NOT IN SOURCES.","i":true},{"text":" Without that line, all four arrangements will quietly fall back on memory when retrieval comes up short, which is precisely when you least want them to."}]},{"t":"h2","id":"keeping-the-sources-small","runs":[{"text":"Keeping the sources small"}]},{"t":"p","runs":[{"text":"A common instinct is to throw everything at it (every policy, every past email, ten years of quotes) on the grounds that more material means better answers."}]},{"t":"p","runs":[{"text":"It does not. A bigger library means retrieval has more chances to fetch something that merely resembles the answer, and it means more opportunities for a stale document to win. Businesses that get good results from grounding usually have a small, curated set: the current price list, the current terms, the current procedures, and nothing superseded."}]},{"t":"p","runs":[{"text":"Delete or archive the old versions rather than leaving them in the folder marked “old”. A file named "},{"text":"prices-2024-OLD.pdf","code":true},{"text":" is, to a retrieval system, just a document about prices."}]},{"t":"h2","id":"a-worked-example","runs":[{"text":"A worked example"}]},{"t":"p","runs":[{"text":"A guesthouse put six documents into a notebook tool: rates, house rules, directions, a cancellation policy, a local-recommendations page and an FAQ built from real emails. Total, about nine pages."}]},{"t":"p","runs":[{"text":"Answers to guest questions became reliable enough to draft from, and the owner could see which page each answer came from. The whole set-up took an afternoon, cost nothing beyond a free tier, and the maintenance is editing a document when something changes, which she was doing anyway."}]}],"check":{"question":[{"text":"A business grounds its assistant on a folder of policy documents. Answers become far more accurate, but one reply confidently states a 30-day return window that no document contains. What is the most useful conclusion?"}],"options":[{"text":[{"text":"Grounding has failed and the folder should be rebuilt, since a single fabrication indicates the retrieval index is corrupt."}],"correct":false,"tempting":[{"text":"Escalates one failure into an infrastructure problem; retrieval that answers most questions correctly is working, and rebuilding it addresses nothing about this reply."}]},{"text":[{"text":"The 30-day figure is probably an industry norm supplied from the model’s memory, so the source folder should be re-checked to see whether returns are covered anywhere in the documents at all."}],"correct":false,"tempting":[{"text":"Identifies the likely origin and stops one step short: the immediate lesson is about the missing instruction, not only about a gap in the documents."}]},{"text":[{"text":"Grounding is unsuitable for policy questions because policies contain exceptions that retrieval cannot represent."}],"correct":false,"tempting":[{"text":"Rejects the technique over a general worry about exceptions, when the reported problem is a fabricated number rather than a mishandled edge case."}]},{"text":[{"text":"Retrieval found nothing relevant and the model fell back on memory, which means the answer-only-from-sources instruction is missing or too weak."}],"correct":true,"tempting":null}],"order":[1,2,0,3],"why":null},"prev":{"slug":"asking-your-own-documents","title":"What actually happens when you upload a file","minutes":9},"next":{"slug":"embeddings-for-non-programmers","title":"Why it finds “refund” when the customer typed “money back”","minutes":10}}}