{"course":{"slug":"ai-for-teachers","title":"AI for Teachers","url":"https://addaly.com/learn/ai-for-teachers","level":"beginner","track":"work"},"lesson":{"slug":"a-student-project-that-teaches-the-mechanism","title":"A student project that teaches the mechanism","url":"https://addaly.com/learn/ai-for-teachers/a-student-project-that-teaches-the-mechanism","position":47,"of":57,"minutes":10,"answer":"Students aged 14 and up can build a working next-word predictor by hand from a page of text (a tally table of which word follows which) and generate sentences from it.","keyIdea":"Students aged 14 and up can build a working next-word predictor by hand from a page of text (a tally table of which word follows which) and generate sentences from it, and the moment their own tiny model produces fluent nonsense is the moment the big one stops being magic."},"body":{"format":"markdown","text":"## Why build one\n\nEverything in the offline lesson can be taught in a period. This project takes two weeks and it does something the activities cannot: students build the thing, run it, watch it produce fluent nonsense, and understand from the inside why a machine that predicts the next word can write a paragraph and still not know anything. It is the single most effective AI lesson for students aged 14 and over, and the paper version needs only a page of text and a tally sheet.\n\n## Week one: the tally\n\nTake one page of a story, about 300 words. Each student, or pair, takes a paragraph. For every word, they record which word came next. \"The\" was followed by \"old\" twice, \"market\" once, \"sun\" once. By the end of the paragraph they have a table: each word, and a tally of every word that followed it.\n\nCombine the paragraphs into one class table on the wall, or in a shared spreadsheet. Three hundred words produce a table of perhaps two hundred entries. It took a class an hour, and they have built a bigram model (a model of which word follows which) from data.\n\n## Week one, continued: generating\n\nNow run it. Pick a starting word. Look up its row. Choose the next word by the tallies: roll a die, or pick the most common, or draw from a bag with one slip per tally mark. Write it down. Look up that word's row. Repeat for twenty words.\n\nRead the sentences aloud. They start well (\"The old woman walked slowly\") because those pairs were in the text. Then they drift: \"slowly into the market was full of the sun was old woman walked\". Grammatical in pieces, meaningless as a whole. The class will laugh. Then ask why.\n\nThe answers come fast, and they are the whole of the course's first block, discovered rather than told. It only looks one word back. It has only seen one page. It has no idea what it is saying; it is following tallies. And a student will ask: \"So the real one is just this, but bigger?\" Yes. Vastly bigger (trillions of words, thousands of words of context instead of one, and a much cleverer way of storing the tallies) and the same idea. The check question's drift is the mechanism made visible.\n\n## Week two: making it better, and seeing why that helps\n\nGive them the levers and let them pull.\n\n**More text.** Add three more pages. The table grows; the sentences hold together for longer. They see that data is what it eats.\n\n**Longer context.** Tally pairs of words and what follows the pair: \"the old\" is followed by \"woman\". Generation from pairs is markedly more fluent, and the table is far larger. They see the trade: context costs memory.\n\n**Temperature.** Instead of drawing from the bag, always pick the most common next word. The output becomes repetitive and loops. Draw more randomly and it becomes wilder. They have discovered the dial from the \"same question, different answer\" lesson by turning it.\n\n**Bias.** Build the table from a page where every doctor is \"he\" and every nurse is \"she\". Generate from \"the doctor\". Ask what happened and why. The retelling activity from the offline lesson taught this in ten minutes; here they have built the mechanism that does it.\n\n## The free path with a laptop\n\nIf any Python is available (Google Colab in a browser, or an online Python environment on a phone) the whole tally can be built in a dozen lines and the class can feed it a whole book from Project Gutenberg. The code is short enough to read:\n\n```python\nimport random\nwords = open(\"story.txt\").read().split()\ntable = {}\nfor a, b in zip(words, words[1:]):\n    table.setdefault(a, []).append(b)\nw = \"The\"\nout = [w]\nfor _ in range(30):\n    w = random.choice(table.get(w, words))\n    out.append(w)\nprint(\" \".join(out))\n```\n\nStudents who have done the paper version read this and recognise every line. That recognition (the code is the tally) is worth more than a term of programming without the paper version first.\n\n## Assessment\n\nNot the model. The explanation. Each student writes a page: how the model works, why the sentences drift, what changed when the context grew, and one thing the real model has that theirs does not. A student who can write that page has an understanding of language models that would pass an interview, and it came from a tally sheet.\n\n## The honest limit\n\nA bigram model is a real model and a very old one; the modern systems differ in ways that matter (attention over thousands of words, learned representations rather than literal tables, training that adjusts billions of numbers) and a student should be told that the project shows the principle, not the architecture. For students who want the next step, the course on how language models work on this site picks up exactly there. For the rest, the principle is the thing they needed, and they have it in their hands."},"check":{"question":"Students build a word-pair tally from one page of a story and use it to generate sentences. The sentences are grammatical for a few words and then drift into nonsense. What has the project shown them, and what would improve the output?","options":[{"text":"The project failed because one page is far too little text to learn from, and it should be repeated from scratch with a whole book.","correct":false,"tempting":"More text would improve it, but calling the drift a failure misses the lesson: the drift is the mechanism, and seeing it is the point."},{"text":"That their model is broken in some way and the real models must work on an entirely different principle to produce coherent text.","correct":false,"tempting":"Real models use the same principle at vastly larger scale with longer context; the project shows the principle, not a different one."},{"text":"That short context gives local fluency and global drift; more text and longer context would help.","correct":true},{"text":"That language cannot really be generated from statistics alone, which is why the real AI systems must use grammar rules underneath.","correct":false,"tempting":"Real models are statistical, not rule-based; the project demonstrates that statistics does generate language, imperfectly at small scale."}]},"prev":{"slug":"the-ai-companion-problem","title":"The AI companion problem","url":"https://addaly.com/learn/ai-for-teachers/the-ai-companion-problem"},"next":{"slug":"the-evidence-on-ai-tutors","title":"The evidence on AI tutors","url":"https://addaly.com/learn/ai-for-teachers/the-evidence-on-ai-tutors"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'A student project that teaches the mechanism', in AI for Teachers. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-teachers/a-student-project-that-teaches-the-mechanism (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"A student project that teaches the mechanism","courseTitle":"AI for Teachers","module":{"index":6,"slug":"teaching-ai-itself","title":"Teaching AI itself"},"answer":null,"keyIdea":"Students aged 14 and up can build a working next-word predictor by hand from a page of text (a tally table of which word follows which) and generate sentences from it, and the moment their own tiny model produces fluent nonsense is the moment the big one stops being magic.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-build-one","runs":[{"text":"Why build one"}]},{"t":"p","runs":[{"text":"Everything in the offline lesson can be taught in a period. This project takes two weeks and it does something the activities cannot: students build the thing, run it, watch it produce fluent nonsense, and understand from the inside why a machine that predicts the next word can write a paragraph and still not know anything. It is the single most effective AI lesson for students aged 14 and over, and the paper version needs only a page of text and a tally sheet."}]},{"t":"h2","id":"week-one-the-tally","runs":[{"text":"Week one: the tally"}]},{"t":"p","runs":[{"text":"Take one page of a story, about 300 words. Each student, or pair, takes a paragraph. For every word, they record which word came next. “The” was followed by “old” twice, “market” once, “sun” once. By the end of the paragraph they have a table: each word, and a tally of every word that followed it."}]},{"t":"p","runs":[{"text":"Combine the paragraphs into one class table on the wall, or in a shared spreadsheet. Three hundred words produce a table of perhaps two hundred entries. It took a class an hour, and they have built a bigram model (a model of which word follows which) from data."}]},{"t":"h2","id":"week-one-continued-generating","runs":[{"text":"Week one, continued: generating"}]},{"t":"p","runs":[{"text":"Now run it. Pick a starting word. Look up its row. Choose the next word by the tallies: roll a die, or pick the most common, or draw from a bag with one slip per tally mark. Write it down. Look up that word’s row. Repeat for twenty words."}]},{"t":"p","runs":[{"text":"Read the sentences aloud. They start well (“The old woman walked slowly”) because those pairs were in the text. Then they drift: “slowly into the market was full of the sun was old woman walked”. Grammatical in pieces, meaningless as a whole. The class will laugh. Then ask why."}]},{"t":"p","runs":[{"text":"The answers come fast, and they are the whole of the course’s first block, discovered rather than told. It only looks one word back. It has only seen one page. It has no idea what it is saying; it is following tallies. And a student will ask: “So the real one is just this, but bigger?” Yes. Vastly bigger (trillions of words, thousands of words of context instead of one, and a much cleverer way of storing the tallies) and the same idea. The check question’s drift is the mechanism made visible."}]},{"t":"h2","id":"week-two-making-it-better-and-seeing-why-that-helps","runs":[{"text":"Week two: making it better, and seeing why that helps"}]},{"t":"p","runs":[{"text":"Give them the levers and let them pull."}]},{"t":"p","runs":[{"text":"More text.","b":true},{"text":" Add three more pages. The table grows; the sentences hold together for longer. They see that data is what it eats."}]},{"t":"p","runs":[{"text":"Longer context.","b":true},{"text":" Tally pairs of words and what follows the pair: “the old” is followed by “woman”. Generation from pairs is markedly more fluent, and the table is far larger. They see the trade: context costs memory."}]},{"t":"p","runs":[{"text":"Temperature.","b":true},{"text":" Instead of drawing from the bag, always pick the most common next word. The output becomes repetitive and loops. Draw more randomly and it becomes wilder. They have discovered the dial from the “same question, different answer” lesson by turning it."}]},{"t":"p","runs":[{"text":"Bias.","b":true},{"text":" Build the table from a page where every doctor is “he” and every nurse is “she”. Generate from “the doctor”. Ask what happened and why. The retelling activity from the offline lesson taught this in ten minutes; here they have built the mechanism that does it."}]},{"t":"h2","id":"the-free-path-with-a-laptop","runs":[{"text":"The free path with a laptop"}],"role":"free"},{"t":"p","runs":[{"text":"If any Python is available (Google Colab in a browser, or an online Python environment on a phone) the whole tally can be built in a dozen lines and the class can feed it a whole book from Project Gutenberg. The code is short enough to read:"}]},{"t":"code","lang":"python","label":"Python","text":"import random\nwords = open(\"story.txt\").read().split()\ntable = {}\nfor a, b in zip(words, words[1:]):\n    table.setdefault(a, []).append(b)\nw = \"The\"\nout = [w]\nfor _ in range(30):\n    w = random.choice(table.get(w, words))\n    out.append(w)\nprint(\" \".join(out))"},{"t":"p","runs":[{"text":"Students who have done the paper version read this and recognise every line. That recognition (the code is the tally) is worth more than a term of programming without the paper version first."}]},{"t":"h2","id":"assessment","runs":[{"text":"Assessment"}]},{"t":"p","runs":[{"text":"Not the model. The explanation. Each student writes a page: how the model works, why the sentences drift, what changed when the context grew, and one thing the real model has that theirs does not. A student who can write that page has an understanding of language models that would pass an interview, and it came from a tally sheet."}]},{"t":"h2","id":"the-honest-limit","runs":[{"text":"The honest limit"}],"role":"limit"},{"t":"p","runs":[{"text":"A bigram model is a real model and a very old one; the modern systems differ in ways that matter (attention over thousands of words, learned representations rather than literal tables, training that adjusts billions of numbers) and a student should be told that the project shows the principle, not the architecture. For students who want the next step, the course on how language models work on this site picks up exactly there. For the rest, the principle is the thing they needed, and they have it in their hands."}]}],"check":{"question":[{"text":"Students build a word-pair tally from one page of a story and use it to generate sentences. The sentences are grammatical for a few words and then drift into nonsense. What has the project shown them, and what would improve the output?"}],"options":[{"text":[{"text":"The project failed because one page is far too little text to learn from, and it should be repeated from scratch with a whole book."}],"correct":false,"tempting":[{"text":"More text would improve it, but calling the drift a failure misses the lesson: the drift is the mechanism, and seeing it is the point."}]},{"text":[{"text":"That their model is broken in some way and the real models must work on an entirely different principle to produce coherent text."}],"correct":false,"tempting":[{"text":"Real models use the same principle at vastly larger scale with longer context; the project shows the principle, not a different one."}]},{"text":[{"text":"That short context gives local fluency and global drift; more text and longer context would help."}],"correct":true,"tempting":null},{"text":[{"text":"That language cannot really be generated from statistics alone, which is why the real AI systems must use grammar rules underneath."}],"correct":false,"tempting":[{"text":"Real models are statistical, not rule-based; the project demonstrates that statistics does generate language, imperfectly at small scale."}]}],"order":[1,2,3,0],"why":null},"prev":{"slug":"the-ai-companion-problem","title":"The AI companion problem","minutes":9},"next":{"slug":"the-evidence-on-ai-tutors","title":"The evidence on AI tutors","minutes":10}}}