Giving the model your own data

What the model does not know

A hosted model has read a great deal of the public internet and none of your database. “Make the assistant know about my customers” cannot mean teaching it; training on your data is expensive, slow, and produces a model that recites approximations. What it means in practice is simpler: at the moment of the question, your code fetches the relevant rows and puts them in the prompt, and the model reads them the way it would read anything else pasted into a message.

That is the whole technique. The engineering is in the word relevant, and in a boundary that must never move.

Level one: the rows you already know

Most questions are about something with an id. “Summarise this customer’s history”: the customer id is in the URL, the session says which user is asking, and a query returns the twenty rows that matter:

const rows = await db.select().from(bookings)
  .where(and(eq(bookings.userId, session.userId), eq(bookings.customerId, customerId)))
  .orderBy(desc(bookings.startsAt)).limit(20);

Format them as compact text, one line per row, the fields that matter, and put that in the user message beneath the question. This is retrieval without a search, it covers more features than people expect, and it is safe by construction because the query is scoped the same way every query in the app is scoped.

Level two: search by words

“Which of my notes mention the roof?” needs a search. Postgres has full-text search built in (to_tsvector and to_tsquery) and SQLite has FTS5. Both find rows containing the words, ranked, in milliseconds, with no new service. Fetch the top ten, paste them, ask. For a small app this is often the right stopping point.

Word search fails when the words differ: “roof” does not find “ceiling leak”, and “cancelled” does not find “called it off”. That is the gap the next level exists for.

Level three: search by meaning

An embedding model turns a piece of text into a list of numbers, typically several hundred to a few thousand of them, arranged so that texts with similar meaning produce lists that point in similar directions. “Roof repair” and “ceiling leak” land close together; “roof repair” and “invoice overdue” do not.

Similarity is measured by the angle between the two lists, called cosine similarity: 1 for identical direction, 0 for unrelated. The search is: embed the question, compare with the stored embedding of every note, take the closest ten.

-- pgvector, an extension available on Neon and Supabase free tiers
SELECT id, body, 1 - (embedding <=> $1) AS similarity
FROM notes
WHERE user_id = $2                    -- the boundary, before ranking
ORDER BY embedding <=> $1
LIMIT 10;

The free path is complete. pgvector stores and compares the vectors inside Postgres. The embeddings themselves come from a small model that runs on your own machine through Ollama, or in the browser or on the server through transformers.js, at zero cost; hosted embedding endpoints are cheap and give somewhat better results. A few thousand notes embed in minutes and re-embed only when they change.

The dedicated treatment of embeddings (what the numbers are, why the directions mean anything, where they fail) belongs to the machine-learning course. For the app builder, the working model is: a vector is a position in a space where nearby means similar, and a similarity search returns neighbours.

The boundary that must not move

Look at the WHERE user_id = $2 above. It runs before the ranking, not after. Filtering after (search everything, then drop the rows that are not this user’s) means other users’ rows were considered, and one of them was the closest match at some point. The question in this lesson’s check is that mistake, and it is common because the generated version builds the search first and adds the user later as an afterthought.

The rule from the injection lesson: the model only sees what this session may see. Retrieval is where that is enforced, in the query, and it is not enforceable anywhere later.

How much to paste

A context window is large and not infinite, and every pasted token is paid for and slows the answer. Ten to twenty rows, trimmed to the fields that matter, is the usual sweet spot. Pasting the whole table because it fits is a habit that works at a hundred rows and produces a $2 call and a confused answer at ten thousand.

The model still answers from itself

Given ten relevant rows and a question, the model answers from the rows most of the time, and sometimes from what it remembers about the world, and it does not mark which. Tell it in the prompt to answer only from the provided records and to say when they do not contain the answer. Then test with a question the rows cannot answer, and see whether it says so or invents. That test is the difference between a feature and a liability, and it is the subject of the next lesson.