Python, from Zero, for AI · Chapter 09

A conversation is a list you pay to resend

1:39 · Narrated with a synthetic voice. Captions are available in the player.

Read the transcript

The last module assembles everything into one working tool. It starts with a chat loop, and with the fact that surprises everybody the first time they see a bill: the model remembers nothing between calls. The conversation exists only as the list you send back in full every turn, so the fortieth question carries the cost of the whole conversation so far. Trim from the oldest turn, keep the system message, and the line goes flat. Then a cache on a small database file, so a script you have run twenty times today has cost you one run. Then the thing that makes all of it worth having: search over documents that belong to you. Chunk them on paragraph boundaries, measured in tokens, with a little overlap so a fact on the boundary survives. Embed each chunk once with a free model on your own processor. Save the matrix and the sources. Embed the question with the same model, because two models put their vectors in unrelated spaces and the scores will look perfectly plausible and mean nothing. Take the best few and hand them to the model with instructions to answer only from those passages and to cite them. Sixty lines, all of it from earlier in the course.

Sources and further reading (1)