Python, from Zero, for AILesson 80 of 89

Module 9 · Building the thing: a small AI program, end to end

A chat loop with memory: keeping history, trimming it, and stopping cleanly

In this lesson

  • 9 min read
  • 8 reading sections
  • Knowledge check
  • Lesson 1 of 10 in this chapter

This chapter’s goalAssemble a complete tool from the course’s parts (a chat loop with bounded history, injection-resistant prompt templates, a sqlite response cache, a document chunker, a local embedding search over your own notes, a free local model as the fallback provider) expose it through a Gradio interface and a FastAPI endpoint, run it on a schedule without double-processing, and gate every change behind a GitHub Actions job that runs the tests.

This chapter’s media

The film explains chapter ideas. The audiobook narrates the complete teaching text.

FormatEnglishहिंदी
Chapter filmWatch1:39Not available
Full audiobookNot availableNot available
Input tokens sent on each turn of one conversation0500010000140Turn numberInput tokens in the request—— Full history, nothing trimmed– – Trimmed to a 4,000-token budgetThe provider remembers nothing between calls, so the history is re-sent and re-billed every turn: turnforty pays for the whole conversation so far, and the reply is the cheap part. Trimming from theoldest user turn, keeping the system message and never leaving a tool call without its result, holdsthe line flat.
The provider remembers nothing between calls, so the history is re-sent and re-billed every turn: turn forty pays for the whole conversation so far, and the reply is the cheap part. Trimming from the oldest user turn, keeping the system message and never leaving a tool call without its result, holds the line flat.

Figure 9.1

A little at a time

Explore an idea, try it, then check what stays.

Step 1 of 10Free · no account needed
Lesson steps

The loop

Python
def chat(provider, system):
    messages = [{"role": "system", "content": system}]
    while True:
        try:
            text = input("you> ").strip()
        except (EOFError, KeyboardInterrupt):
            print()
            break
        if not text:
            continue
        if text in ("/quit", "/exit"):
            break
        messages.append({"role": "user", "content": text})
        reply = provider.complete(messages)
        messages.append({"role": "assistant", "content": reply})
        print(f"bot> {reply}\n")
    return messages

Every piece of this you have written before: a while True with a break, input(), a list of dicts, a provider with one method. The two lines that make it a conversation rather than a series of unrelated questions are the two appends. The model sees the full list every turn. That is its memory. There is no other.

Your place and first choices stay on this browser. Classroom credit is recorded separately.

Stuck on something?Ask Divya, your AI tutor

How to cite: Addaly (2026) ‘A chat loop with memory: keeping history, trimming it, and stopping cleanly’, in Python, from Zero, for AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/python-for-ai/a-chat-loop-with-memory (Accessed: 21 November 2026).