{"course":{"slug":"hugging-face","title":"Hugging Face, End to End","url":"https://addaly.com/learn/hugging-face","level":"intermediate","track":"create"},"lesson":{"slug":"a-search-index-without-a-database","title":"NumPy first, FAISS second, a database rarely","url":"https://addaly.com/learn/hugging-face/a-search-index-without-a-database","position":53,"of":75,"minutes":9,"answer":"Exact search with one matrix multiply handles a million vectors in tens of milliseconds.","keyIdea":"Exact search with one matrix multiply handles a million vectors in tens of milliseconds, so NumPy is the right first index and FAISS or a vector database earns its place only at genuine scale or when filtered, updating, shared indexes are needed, and adding BM25 alongside fixes what embeddings are worst at."},"body":{"format":"markdown","text":"## The version that is enough for most projects\n\n```python\nimport numpy as np, json\nfrom sentence_transformers import SentenceTransformer\n\nmodel = SentenceTransformer(\"BAAI/bge-small-en-v1.5\")\nV = model.encode(chunks, normalize_embeddings=True, batch_size=64).astype(\"float32\")\nnp.save(\"vectors.npy\", V)\njson.dump(meta, open(\"meta.json\", \"w\"))          # same order as V\n\ndef search(q, k=5):\n    qv = model.encode(q, normalize_embeddings=True, prompt_name=\"query\")\n    scores = V @ qv\n    idx = np.argpartition(-scores, k)[:k]\n    idx = idx[np.argsort(-scores[idx])]\n    return [(float(scores[i]), meta[i]) for i in idx]\n```\n\nThat is a complete semantic search engine in twelve lines, with no server and no dependency beyond NumPy. On 100,000 vectors of 384 dimensions it searches in a handful of milliseconds. On a million it is tens of milliseconds, which is still fine behind a web request.\n\n`argpartition` rather than a full sort matters at scale: it finds the top k without ordering everything, and on a million items that is several times faster.\n\n## When NumPy stops being enough\n\nTwo thresholds, and they are further away than people assume:\n\n- **Memory.** A million 384-dimensional float32 vectors is 1.5 GB. Ten million is 15 GB, and now you have a problem.\n- **Latency.** Exact search is linear in corpus size. Beyond a few million, milliseconds become hundreds of milliseconds.\n\nBelow those, exact search is not only sufficient: it is *better*, because it is exact. Every approximate index trades recall for speed, and paying that price before you need it is a common and avoidable mistake.\n\n## FAISS, when you do cross the line\n\n```bash\npip install faiss-cpu\n```\n\n```python\nimport faiss\n\nindex = faiss.IndexFlatIP(V.shape[1])     # exact inner product\nindex.add(V)\nD, I = index.search(qv.reshape(1, -1), 5)\n```\n\n`IndexFlatIP` is still exact: FAISS with a flat index is a faster matrix multiply, not an approximation, and it is a sensible first step. Approximation comes when you ask for it:\n\n```python\nindex = faiss.IndexHNSWFlat(V.shape[1], 32)   # graph index, approximate\nindex.hnsw.efConstruction = 200\nindex.add(V)\nindex.hnsw.efSearch = 64                      # higher = better recall, slower\n```\n\nHNSW builds a navigable graph and walks it. It is fast, it uses more memory than the raw vectors, and `efSearch` is the dial between recall and latency. **Measure the recall you are losing** by comparing against exact search on a few hundred queries: approximate indexes are routinely deployed with nobody knowing whether they return 99% or 80% of the right answers.\n\n`IndexIVFPQ` compresses vectors as well, trading more accuracy for much less memory. It is what you reach for at tens of millions of vectors on one machine.\n\n## Vector databases, and when they are actually warranted\n\nQdrant, Weaviate, Milvus, Chroma, pgvector and the rest give you persistence, metadata filtering, incremental updates, concurrency and replication. Those are real features and none of them is about search quality.\n\nReach for one when you need:\n\n- **Filtered search**: \"nearest neighbours where `language = 'hi'` and `date > 2024`\". Doing this properly is harder than it looks, because filtering after retrieval can leave you with two results out of five.\n- **Frequent updates**: documents added and deleted continuously, without rebuilding.\n- **More vectors than fit on one machine.**\n- **Several services sharing one index.**\n\nIf you already run PostgreSQL, `pgvector` is usually the right answer: your metadata and your vectors live in one place, filtering is ordinary SQL, and you add no new operational surface. That is a free option and it is under-recommended because it is not exciting.\n\n## Keyword search is not obsolete\n\nEmbeddings are bad at exact identifiers: a part number, an error code, a surname. BM25 (classic keyword ranking) handles those perfectly and runs in-process:\n\n```bash\npip install rank_bm25\n```\n\nCombining both and merging the ranked lists (reciprocal rank fusion is a four-line function) reliably beats either alone, typically by a wide margin on real corpora with codes and names in them. Hybrid retrieval is not an advanced technique; it is the default that most systems should start with.\n\n## Do this now\n\nBuild the NumPy index over your own documents and search it. Then add BM25 over the same chunks and compare the top five for three queries containing a specific code or name. The gap you see is the argument for hybrid search."},"check":{"question":"A team replaces exact NumPy search over 80,000 chunks with an HNSW index and finds quality has dropped slightly, with no speed problem before or after. What went wrong in the decision?","options":[{"text":"HNSW requires unnormalised vectors, so the switch silently changed the similarity being computed.","correct":false,"tempting":"Index type and normalisation are independent choices made by the caller."},{"text":"The graph parameters were left at defaults, and tuning `efConstruction` would restore exact results.","correct":false,"tempting":"Tuning improves recall but an approximate index does not become exact at any setting."},{"text":"They adopted an approximate index below the scale where exactness cost anything.","correct":true},{"text":"HNSW indexes must be rebuilt after every query batch, so the index had drifted out of date with the corpus.","correct":false,"tempting":"Invents a rebuild requirement; queries do not modify the graph."}]},"prev":{"slug":"chunking-text-for-retrieval","title":"Chunking decides what your search can find","url":"https://addaly.com/learn/hugging-face/chunking-text-for-retrieval"},"next":{"slug":"reranking-with-cross-encoders","title":"The second pass that fixes the first","url":"https://addaly.com/learn/hugging-face/reranking-with-cross-encoders"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'NumPy first, FAISS second, a database rarely', in Hugging Face, End to End. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/hugging-face/a-search-index-without-a-database (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"NumPy first, FAISS second, a database rarely","courseTitle":"Hugging Face, End to End","module":{"index":6,"slug":"embeddings-and-retrieval","title":"Embeddings, and search that understands"},"answer":null,"keyIdea":"Exact search with one matrix multiply handles a million vectors in tens of milliseconds, so NumPy is the right first index and FAISS or a vector database earns its place only at genuine scale or when filtered, updating, shared indexes are needed, and adding BM25 alongside fixes what embeddings are worst at.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-version-that-is-enough-for-most-projects","runs":[{"text":"The version that is enough for most projects"}]},{"t":"code","lang":"python","label":"Python","text":"import numpy as np, json\nfrom sentence_transformers import SentenceTransformer\n\nmodel = SentenceTransformer(\"BAAI/bge-small-en-v1.5\")\nV = model.encode(chunks, normalize_embeddings=True, batch_size=64).astype(\"float32\")\nnp.save(\"vectors.npy\", V)\njson.dump(meta, open(\"meta.json\", \"w\"))          # same order as V\n\ndef search(q, k=5):\n    qv = model.encode(q, normalize_embeddings=True, prompt_name=\"query\")\n    scores = V @ qv\n    idx = np.argpartition(-scores, k)[:k]\n    idx = idx[np.argsort(-scores[idx])]\n    return [(float(scores[i]), meta[i]) for i in idx]"},{"t":"p","runs":[{"text":"That is a complete semantic search engine in twelve lines, with no server and no dependency beyond NumPy. On 100,000 vectors of 384 dimensions it searches in a handful of milliseconds. On a million it is tens of milliseconds, which is still fine behind a web request."}]},{"t":"p","runs":[{"text":"argpartition","code":true},{"text":" rather than a full sort matters at scale: it finds the top k without ordering everything, and on a million items that is several times faster."}]},{"t":"h2","id":"when-numpy-stops-being-enough","runs":[{"text":"When NumPy stops being enough"}]},{"t":"p","runs":[{"text":"Two thresholds, and they are further away than people assume:"}]},{"t":"ul","items":[[{"text":"Memory.","b":true},{"text":" A million 384-dimensional float32 vectors is 1.5 GB. Ten million is 15 GB, and now you have a problem."}],[{"text":"Latency.","b":true},{"text":" Exact search is linear in corpus size. Beyond a few million, milliseconds become hundreds of milliseconds."}]]},{"t":"p","runs":[{"text":"Below those, exact search is not only sufficient: it is "},{"text":"better","i":true},{"text":", because it is exact. Every approximate index trades recall for speed, and paying that price before you need it is a common and avoidable mistake."}]},{"t":"h2","id":"faiss-when-you-do-cross-the-line","runs":[{"text":"FAISS, when you do cross the line"}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install faiss-cpu"},{"t":"code","lang":"python","label":"Python","text":"import faiss\n\nindex = faiss.IndexFlatIP(V.shape[1])     # exact inner product\nindex.add(V)\nD, I = index.search(qv.reshape(1, -1), 5)"},{"t":"p","runs":[{"text":"IndexFlatIP","code":true},{"text":" is still exact: FAISS with a flat index is a faster matrix multiply, not an approximation, and it is a sensible first step. Approximation comes when you ask for it:"}]},{"t":"code","lang":"python","label":"Python","text":"index = faiss.IndexHNSWFlat(V.shape[1], 32)   # graph index, approximate\nindex.hnsw.efConstruction = 200\nindex.add(V)\nindex.hnsw.efSearch = 64                      # higher = better recall, slower"},{"t":"p","runs":[{"text":"HNSW builds a navigable graph and walks it. It is fast, it uses more memory than the raw vectors, and "},{"text":"efSearch","code":true},{"text":" is the dial between recall and latency. "},{"text":"Measure the recall you are losing","b":true},{"text":" by comparing against exact search on a few hundred queries: approximate indexes are routinely deployed with nobody knowing whether they return 99% or 80% of the right answers."}]},{"t":"p","runs":[{"text":"IndexIVFPQ","code":true},{"text":" compresses vectors as well, trading more accuracy for much less memory. It is what you reach for at tens of millions of vectors on one machine."}]},{"t":"h2","id":"vector-databases-and-when-they-are-actually-warranted","runs":[{"text":"Vector databases, and when they are actually warranted"}]},{"t":"p","runs":[{"text":"Qdrant, Weaviate, Milvus, Chroma, pgvector and the rest give you persistence, metadata filtering, incremental updates, concurrency and replication. Those are real features and none of them is about search quality."}]},{"t":"p","runs":[{"text":"Reach for one when you need:"}]},{"t":"ul","items":[[{"text":"Filtered search","b":true},{"text":": “nearest neighbours where "},{"text":"language = 'hi'","code":true},{"text":" and "},{"text":"date > 2024","code":true},{"text":"”. Doing this properly is harder than it looks, because filtering after retrieval can leave you with two results out of five."}],[{"text":"Frequent updates","b":true},{"text":": documents added and deleted continuously, without rebuilding."}],[{"text":"More vectors than fit on one machine.","b":true}],[{"text":"Several services sharing one index.","b":true}]]},{"t":"p","runs":[{"text":"If you already run PostgreSQL, "},{"text":"pgvector","code":true},{"text":" is usually the right answer: your metadata and your vectors live in one place, filtering is ordinary SQL, and you add no new operational surface. That is a free option and it is under-recommended because it is not exciting."}]},{"t":"h2","id":"keyword-search-is-not-obsolete","runs":[{"text":"Keyword search is not obsolete"}]},{"t":"p","runs":[{"text":"Embeddings are bad at exact identifiers: a part number, an error code, a surname. BM25 (classic keyword ranking) handles those perfectly and runs in-process:"}]},{"t":"code","lang":"bash","label":"Bash","text":"pip install rank_bm25"},{"t":"p","runs":[{"text":"Combining both and merging the ranked lists (reciprocal rank fusion is a four-line function) reliably beats either alone, typically by a wide margin on real corpora with codes and names in them. Hybrid retrieval is not an advanced technique; it is the default that most systems should start with."}]},{"t":"h2","id":"do-this-now","runs":[{"text":"Do this now"}],"role":"try"},{"t":"p","runs":[{"text":"Build the NumPy index over your own documents and search it. Then add BM25 over the same chunks and compare the top five for three queries containing a specific code or name. The gap you see is the argument for hybrid search."}]}],"check":{"question":[{"text":"A team replaces exact NumPy search over 80,000 chunks with an HNSW index and finds quality has dropped slightly, with no speed problem before or after. What went wrong in the decision?"}],"options":[{"text":[{"text":"HNSW requires unnormalised vectors, so the switch silently changed the similarity being computed."}],"correct":false,"tempting":[{"text":"Index type and normalisation are independent choices made by the caller."}]},{"text":[{"text":"The graph parameters were left at defaults, and tuning "},{"text":"efConstruction","code":true},{"text":" would restore exact results."}],"correct":false,"tempting":[{"text":"Tuning improves recall but an approximate index does not become exact at any setting."}]},{"text":[{"text":"They adopted an approximate index below the scale where exactness cost anything."}],"correct":true,"tempting":null},{"text":[{"text":"HNSW indexes must be rebuilt after every query batch, so the index had drifted out of date with the corpus."}],"correct":false,"tempting":[{"text":"Invents a rebuild requirement; queries do not modify the graph."}]}],"order":[1,0,3,2],"why":null},"prev":{"slug":"chunking-text-for-retrieval","title":"Chunking decides what your search can find","minutes":9},"next":{"slug":"reranking-with-cross-encoders","title":"The second pass that fixes the first","minutes":8}}}