{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"retrieval-for-agents","title":"Grep first, embeddings later","url":"https://addaly.com/learn/agents-and-automation/retrieval-for-agents","position":26,"of":72,"minutes":9,"answer":"An agent that can search repeatedly needs a good search tool more than it needs a clever index, and lexical search beats vector search on exactly the identifiers that matter most.","keyIdea":"An agent that can search repeatedly needs a good search tool more than it needs a clever index, and lexical search beats vector search on exactly the identifiers that matter most."},"body":{"format":"markdown","text":"## The agent case is not the chatbot case\n\nRetrieval for a chatbot is one shot: fetch the best passages you can and hope the answer is in them. An agent gets to search again. It can read a result, notice the part number it needed, and search for that. It can widen a date range, try a different spelling, open the file the snippet came from.\n\nThat changes what you should build. The clever part is not the index, it is the **tool surface**: what the agent can ask for, and what comes back. A retrieval tool with filters, a result count, snippets with line numbers and a way to fetch more of a document will outperform a better index behind a worse interface, because the agent can compensate for a mediocre first result and cannot compensate for not being able to ask a second question.\n\n## Lexical search is badly underrated\n\nThe instinct now is to reach for embeddings. For a large share of real work, plain text search is better, and it is better precisely where accuracy matters most.\n\nIdentifiers, error codes, part numbers, SKUs, invoice references, function names, legal clause numbers: these are tokens whose value lies in being exact. A vector model is trained to place similar things together, which means it deliberately blurs the difference between `INV-2026-0041` and `INV-2026-0141`. Lexical search does not blur, and returns the one you asked for.\n\nThe free options are good and boring. `ripgrep` over a directory is instant on hundreds of megabytes. **SQLite FTS5** gives you BM25 ranking in a single file with no server. **PostgreSQL** full-text search is already installed wherever your data is. All three support the filters that make retrieval useful (date ranges, document type, owner) which most vector stores bolt on awkwardly.\n\nGive the agent both, as two tools with honest descriptions: one for \"find this exact string\", one for \"find passages about this idea\". Then let it choose, and watch the traces to see which it actually uses.\n\n## Hybrid, cheaply\n\nWhen you want both, you do not need a framework. Run the BM25 query and the vector query, take the top twenty from each, and combine with reciprocal rank fusion: each document scores the sum of `1 / (60 + rank)` over the lists it appears in. Twenty lines of code, no tuning, and it reliably beats either list alone. The constant 60 is a convention from the original work, not a magic number, and results are not sensitive to it.\n\n## What to return\n\nSnippets, with a handle. A search result should be a few hundred words of context, a document id, a page or line number, and enough for the agent to decide whether to open the whole thing with a separate `read` tool.\n\nReturning whole documents is the most common cause of a run that dies at step nine: three searches with five full documents each will fill any context window, and everything after that is degraded. Cap the result size in the tool, and say in the response how many results were omitted, so the agent knows there is more rather than assuming it has seen everything.\n\n## Permissions belong in the query\n\nIf the agent searches on behalf of a person, the permission filter goes into the query, not into a filtering step afterwards. Anything retrieved into the transcript is available to be summarised into the answer, and a post-filter that removes a document from the result list has already lost if the snippet was read.\n\nThis is the same argument as the one about injection and blast radius: the safe boundary is the one the data never crosses. Pass the caller's identity down to the search tool and let the index enforce it.\n\n## Freshness, and knowing when the index is stale\n\nEvery index is a copy, and a copy is wrong the moment the source changes. Two things to build on day one: store an `indexed_at` beside each document and expose it in results, so the agent can say \"as of Tuesday\"; and record the count of documents indexed per run so a build that indexed nothing is visible instead of appearing to succeed.\n\nAn agent quoting a policy that was superseded last month is a failure nobody attributes to retrieval, because the answer is fluent and the citation is real. Only the timestamp gives it away.\n\n## Evaluate retrieval on its own\n\nIf the final answer is wrong, there are two candidates: the right passage was never retrieved, or it was retrieved and misused. These have completely different fixes, and you cannot tell them apart by reading answers.\n\nBuild a small set of questions with the document that ought to be found for each, and measure whether it appears in the top five. Twenty questions is enough to make the difference visible. Do this before touching the generation prompt, because most \"the model got it wrong\" complaints turn out to be retrieval that never delivered the evidence."},"check":{"question":"An agent must find a specific invoice reference, `INV-2026-0041`, in a document store. Vector search keeps returning invoices with adjacent numbers. What is the mechanism, and what should the tool surface offer?","options":[{"text":"The embedding index is out of date and needs rebuilding so the exact document is present.","correct":false,"tempting":"A stale index explains a missing document, not a set of near-miss references that are clearly being found and ranked."},{"text":"Embeddings are trained to place similar strings together, which deliberately blurs near-identical identifiers, so the agent also needs a lexical search tool.","correct":true},{"text":"The chunks are too large, so the reference is diluted by surrounding text; smaller chunks will isolate it.","correct":false,"tempting":"Smaller chunks raise the reference’s share of the vector but do not restore the distinction between two references that differ by one character."},{"text":"The similarity threshold is too permissive and should be raised until only the exact match survives.","correct":false,"tempting":"Treats a ranking problem as a cut-off problem; the near-miss references genuinely score high, so no threshold separates them cleanly."}]},"prev":{"slug":"deduplication-and-matching","title":"The same customer, three times","url":"https://addaly.com/learn/agents-and-automation/deduplication-and-matching"},"next":{"slug":"embeddings-in-practice","title":"What an embedding actually is","url":"https://addaly.com/learn/agents-and-automation/embeddings-in-practice"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Grep first, embeddings later', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/retrieval-for-agents (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Grep first, embeddings later","courseTitle":"Agents and Automation","module":{"index":3,"slug":"data-plumbing","title":"The data underneath: sources, documents, records"},"answer":null,"keyIdea":"An agent that can search repeatedly needs a good search tool more than it needs a clever index, and lexical search beats vector search on exactly the identifiers that matter most.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-agent-case-is-not-the-chatbot-case","runs":[{"text":"The agent case is not the chatbot case"}]},{"t":"p","runs":[{"text":"Retrieval for a chatbot is one shot: fetch the best passages you can and hope the answer is in them. An agent gets to search again. It can read a result, notice the part number it needed, and search for that. It can widen a date range, try a different spelling, open the file the snippet came from."}]},{"t":"p","runs":[{"text":"That changes what you should build. The clever part is not the index, it is the "},{"text":"tool surface","b":true},{"text":": what the agent can ask for, and what comes back. A retrieval tool with filters, a result count, snippets with line numbers and a way to fetch more of a document will outperform a better index behind a worse interface, because the agent can compensate for a mediocre first result and cannot compensate for not being able to ask a second question."}]},{"t":"h2","id":"lexical-search-is-badly-underrated","runs":[{"text":"Lexical search is badly underrated"}]},{"t":"p","runs":[{"text":"The instinct now is to reach for embeddings. For a large share of real work, plain text search is better, and it is better precisely where accuracy matters most."}]},{"t":"p","runs":[{"text":"Identifiers, error codes, part numbers, SKUs, invoice references, function names, legal clause numbers: these are tokens whose value lies in being exact. A vector model is trained to place similar things together, which means it deliberately blurs the difference between "},{"text":"INV-2026-0041","code":true},{"text":" and "},{"text":"INV-2026-0141","code":true},{"text":". Lexical search does not blur, and returns the one you asked for."}]},{"t":"p","runs":[{"text":"The free options are good and boring. "},{"text":"ripgrep","code":true},{"text":" over a directory is instant on hundreds of megabytes. "},{"text":"SQLite FTS5","b":true},{"text":" gives you BM25 ranking in a single file with no server. "},{"text":"PostgreSQL","b":true},{"text":" full-text search is already installed wherever your data is. All three support the filters that make retrieval useful (date ranges, document type, owner) which most vector stores bolt on awkwardly."}]},{"t":"p","runs":[{"text":"Give the agent both, as two tools with honest descriptions: one for “find this exact string”, one for “find passages about this idea”. Then let it choose, and watch the traces to see which it actually uses."}]},{"t":"h2","id":"hybrid-cheaply","runs":[{"text":"Hybrid, cheaply"}]},{"t":"p","runs":[{"text":"When you want both, you do not need a framework. Run the BM25 query and the vector query, take the top twenty from each, and combine with reciprocal rank fusion: each document scores the sum of "},{"text":"1 / (60 + rank)","code":true},{"text":" over the lists it appears in. Twenty lines of code, no tuning, and it reliably beats either list alone. The constant 60 is a convention from the original work, not a magic number, and results are not sensitive to it."}]},{"t":"h2","id":"what-to-return","runs":[{"text":"What to return"}]},{"t":"p","runs":[{"text":"Snippets, with a handle. A search result should be a few hundred words of context, a document id, a page or line number, and enough for the agent to decide whether to open the whole thing with a separate "},{"text":"read","code":true},{"text":" tool."}]},{"t":"p","runs":[{"text":"Returning whole documents is the most common cause of a run that dies at step nine: three searches with five full documents each will fill any context window, and everything after that is degraded. Cap the result size in the tool, and say in the response how many results were omitted, so the agent knows there is more rather than assuming it has seen everything."}]},{"t":"h2","id":"permissions-belong-in-the-query","runs":[{"text":"Permissions belong in the query"}]},{"t":"p","runs":[{"text":"If the agent searches on behalf of a person, the permission filter goes into the query, not into a filtering step afterwards. Anything retrieved into the transcript is available to be summarised into the answer, and a post-filter that removes a document from the result list has already lost if the snippet was read."}]},{"t":"p","runs":[{"text":"This is the same argument as the one about injection and blast radius: the safe boundary is the one the data never crosses. Pass the caller’s identity down to the search tool and let the index enforce it."}]},{"t":"h2","id":"freshness-and-knowing-when-the-index-is-stale","runs":[{"text":"Freshness, and knowing when the index is stale"}]},{"t":"p","runs":[{"text":"Every index is a copy, and a copy is wrong the moment the source changes. Two things to build on day one: store an "},{"text":"indexed_at","code":true},{"text":" beside each document and expose it in results, so the agent can say “as of Tuesday”; and record the count of documents indexed per run so a build that indexed nothing is visible instead of appearing to succeed."}]},{"t":"p","runs":[{"text":"An agent quoting a policy that was superseded last month is a failure nobody attributes to retrieval, because the answer is fluent and the citation is real. Only the timestamp gives it away."}]},{"t":"h2","id":"evaluate-retrieval-on-its-own","runs":[{"text":"Evaluate retrieval on its own"}]},{"t":"p","runs":[{"text":"If the final answer is wrong, there are two candidates: the right passage was never retrieved, or it was retrieved and misused. These have completely different fixes, and you cannot tell them apart by reading answers."}]},{"t":"p","runs":[{"text":"Build a small set of questions with the document that ought to be found for each, and measure whether it appears in the top five. Twenty questions is enough to make the difference visible. Do this before touching the generation prompt, because most “the model got it wrong” complaints turn out to be retrieval that never delivered the evidence."}]}],"check":{"question":[{"text":"An agent must find a specific invoice reference, "},{"text":"INV-2026-0041","code":true},{"text":", in a document store. Vector search keeps returning invoices with adjacent numbers. What is the mechanism, and what should the tool surface offer?"}],"options":[{"text":[{"text":"The embedding index is out of date and needs rebuilding so the exact document is present."}],"correct":false,"tempting":[{"text":"A stale index explains a missing document, not a set of near-miss references that are clearly being found and ranked."}]},{"text":[{"text":"Embeddings are trained to place similar strings together, which deliberately blurs near-identical identifiers, so the agent also needs a lexical search tool."}],"correct":true,"tempting":null},{"text":[{"text":"The chunks are too large, so the reference is diluted by surrounding text; smaller chunks will isolate it."}],"correct":false,"tempting":[{"text":"Smaller chunks raise the reference’s share of the vector but do not restore the distinction between two references that differ by one character."}]},{"text":[{"text":"The similarity threshold is too permissive and should be raised until only the exact match survives."}],"correct":false,"tempting":[{"text":"Treats a ranking problem as a cut-off problem; the near-miss references genuinely score high, so no threshold separates them cleanly."}]}],"order":[3,0,2,1],"why":null},"prev":{"slug":"deduplication-and-matching","title":"The same customer, three times","minutes":9},"next":{"slug":"embeddings-in-practice","title":"What an embedding actually is","minutes":10}}}