Grounding: answering from your material rather than its memory

Two ways to get an answer

Ask a model what your refund policy is and it will tell you something. Where that something came from is the whole question.

From memory. The model produces the most likely words following your question, drawn from everything it read during training. Fast, free, and unconnected to your business.

From a source. The tool finds relevant text (your policy document, your price list, your past emails) puts it in front of the model, and asks it to answer using that. Slower, sometimes costs a little more, and the answer is about your business.

The second is called grounding, or retrieval-augmented generation when people want it to sound technical. It is the single most useful architectural idea for a small business, and it explains why a tool that has your documents is a different class of thing from one that does not.

Why it works

It changes the task from recall to reading comprehension.

Recall is where models are weakest: they are reconstructing, not retrieving, so a plausible answer and a true one are produced by the same process. Reading comprehension is where they are strongest. Given a passage that contains the answer, extracting and rephrasing it is close to the thing they do best.

That is the whole mechanism, and it explains a pattern you will notice: a model that invents wildly about your business becomes remarkably reliable the moment you paste your actual price list. Nothing about the model changed. The task changed.

What you get beyond accuracy

Three things, and the second is the one that matters most in practice.

Currency. Change the document, and the answer changes. No retraining, no waiting for a new model. Your price rise takes effect the moment you edit the file.

Citations. A grounded system can say which document and which line an answer came from. This converts checking from re-reading everything to glancing at one quoted line. It is the difference between an answer you have to trust and an answer you can verify.

Confinement. With the book closed, questions outside your material get a clean refusal rather than an improvised answer. For anything customer-facing this is the property you actually want.

The failures grounding does not fix

Being honest about this matters, because “we use RAG” gets said as though it settles the question.

Bad sources produce bad answers, confidently. A grounded system quoting last year’s price list is worse than an ungrounded one, because it now has a citation. Everything depends on the documents being right and current.

Retrieval can miss. If the relevant passage is not fetched, the model answers from what it did fetch, and may fall back on memory anyway unless firmly instructed not to.

It can still fabricate a citation. Less often, but it happens: a quoted line that reads perfectly and appears nowhere in the source. Which is why the check is against the document, not against the presence of a quote.

It does not add judgement. The system can tell you your policy says fourteen days. It cannot tell you whether to make an exception for this customer.

Doing this without building anything

You do not need a developer, and most small businesses should not have one for this.

• Paste the relevant page into the chat with your question. Crude, free, and it is real grounding.
• Standing instructions holding your facts sheet. Grounding on your most-used material, permanently.
• A document tool: a project with files attached, or a purpose-built notebook product. Grounding across a small library.
• A local setup if the documents are sensitive: open tools such as AnythingLLM or Open WebUI paired with Ollama do this on your own machine, free.

The instruction that turns any of them into a grounded system is the same one from module two: answer only from the sources below; if it is not there, say NOT IN SOURCES. Without that line, all four arrangements will quietly fall back on memory when retrieval comes up short, which is precisely when you least want them to.

Keeping the sources small

A common instinct is to throw everything at it (every policy, every past email, ten years of quotes) on the grounds that more material means better answers.

It does not. A bigger library means retrieval has more chances to fetch something that merely resembles the answer, and it means more opportunities for a stale document to win. Businesses that get good results from grounding usually have a small, curated set: the current price list, the current terms, the current procedures, and nothing superseded.

Delete or archive the old versions rather than leaving them in the folder marked “old”. A file named prices-2024-OLD.pdf is, to a retrieval system, just a document about prices.

A worked example

A guesthouse put six documents into a notebook tool: rates, house rules, directions, a cancellation policy, a local-recommendations page and an FAQ built from real emails. Total, about nine pages.

Answers to guest questions became reliable enough to draft from, and the owner could see which page each answer came from. The whole set-up took an afternoon, cost nothing beyond a free tier, and the maintenance is editing a document when something changes, which she was doing anyway.