Search over your own notes: chunks, embeddings, cosine, and sixty lines

What you are building

A box you type a question into that finds the passages in your own notes most likely to answer it, and optionally hands those passages to a model to write the answer. It is the pattern under most "chat with your documents" products, and every piece of it is something this course has already built: file walking from module 5, chunking from the previous lesson, a vector matrix and cosine similarity from module 8, a provider from module 7, a cache from two lessons ago.

Embeddings, free

The hosted embedding endpoints are cheap — fractions of a cent per thousand chunks — and a local model is free and runs on a CPU:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")        # ~90 MB download, 384 dimensions
vectors = model.encode(texts, normalize_embeddings=True, batch_size=64, show_progress_bar=True)

all-MiniLM-L6-v2 embeds a few hundred sentences per second on a laptop CPU and is adequate for English notes. For Hindi and other languages, paraphrase-multilingual-MiniLM-L12-v2 is the same size and covers fifty languages. normalize_embeddings=True returns unit vectors, so the dot product is the cosine, as module 8 established. The hosted equivalent is client.embeddings.create(input=texts, model=...) with the vectors in data[i].embedding; normalise them yourself.

Indexing

import json, numpy as np
from pathlib import Path

def build_index(notes_dir, out_dir, max_tokens=300):
    chunks = []
    for path in sorted(Path(notes_dir).rglob("*.md")):
        text = path.read_text(encoding="utf-8")
        for i, c in enumerate(with_overlap(chunk(text, max_tokens, count))):
            chunks.append({"doc": str(path), "index": i, "text": c})
    vectors = model.encode([c["text"] for c in chunks], normalize_embeddings=True)
    np.save(Path(out_dir) / "vectors.npy", vectors.astype(np.float32))
    Path(out_dir, "chunks.json").write_text(json.dumps(chunks, ensure_ascii=False))
    print(f"{len(chunks)} chunks, {vectors.shape}")

Two files: a (n, 384) float32 matrix and a JSON list where row i of the matrix describes chunk i of the list. The correspondence by position is the whole index; keep the two files together and rebuild both or neither.

Encoding is the slow step. Five thousand chunks take a minute or two on a CPU. Cache by content hash, as the sqlite lesson did, so re-indexing after editing one note re-embeds one note.

Querying

def search(query, k=5):
    vectors = np.load("index/vectors.npy")
    chunks = json.loads(Path("index/chunks.json").read_text())
    q = model.encode([query], normalize_embeddings=True)[0]
    sims = vectors @ q
    top = np.argpartition(sims, -k)[-k:]
    top = top[np.argsort(sims[top])[::-1]]
    return [(float(sims[i]), chunks[i]) for i in top]

That is module 8's cosine lesson verbatim. Load the two files once at startup, not per query. Print the results with their scores and sources:

for score, c in search("how do I rotate the API key"):
    print(f"{score:.3f}  {c['doc']}#{c['index']}\n   {c['text'][:120]}...")

The scores are worth watching. A top result at 0.3 when good matches score 0.6 means nothing relevant exists, and the honest answer is "not found", not the best of a bad set.

The rule that must not be broken

The query must be embedded with the same model that embedded the chunks. Two models produce vectors in unrelated spaces; even at the same dimension, a query from one against chunks from another gives similarities that are noise. Record the model name in the index folder, check it at query time, and treat a change of embedding model as a full rebuild. This is the single most common way a working search silently breaks.

Answering with a model

def answer(question, k=5):
    hits = search(question, k)
    context = "\n\n".join(f"[{i+1}] ({c['doc']})\n{c['text']}" for i, (s, c) in enumerate(hits))
    prompt = render("answer_from_notes", question=question, context=context)
    return provider.complete([{"role": "user", "content": prompt}]), hits

The template tells the model to answer from the numbered passages and to cite them by number, and to say so if the passages do not contain the answer. The user sees the answer and the sources beneath it — which is what makes the answer checkable, and what separates this from a model guessing. Everything about how well this works — chunk size, how many passages, whether to rerank, how to evaluate — belongs to the context-engineering and evaluation courses. The Python does not change.

Limits you should state

Brute-force search over a matrix is exact and fast up to a few hundred thousand chunks, per module 8's arithmetic. Embeddings capture meaning approximately: a question phrased very differently from the note may not match, and a note that mentions the words without answering the question may rank high. Exact identifiers — an error code, a name — are often better found with plain text search, and a good tool does both and merges. A small local model will be worse than a hosted one on nuanced queries, and better than nothing by a wide margin.

Try this now

Point build_index at a folder of your own notes or a project's docs, search for five things you know are in there, and look at the scores. Then re-embed the query with a different model and watch the results degrade. Put the model name in the index and make search refuse to run against the wrong one.