{"course":{"slug":"ai-for-students","title":"AI for Students","url":"https://addaly.com/learn/ai-for-students","level":"beginner","track":"work"},"lesson":{"slug":"what-a-model-actually-does","title":"What it is actually doing","url":"https://addaly.com/learn/ai-for-students/what-a-model-actually-does","position":3,"of":73,"minutes":9,"answer":"A language model predicts the next token from a compressed statistical impression of its training text, so it is strong on repeated general material, weak on specific rare facts.","keyIdea":"A language model predicts the next token from a compressed statistical impression of its training text, so it is strong on repeated general material, weak on specific rare facts, and blind to the letters inside words."},"body":{"format":"markdown","text":"## One token at a time\n\nA language model does one thing. Given the text so far, it produces a probability for every possible next chunk of text, and one chunk is chosen. Then it does it again, with that chunk added to the text. Then again. An answer that took you fifteen seconds to read was built in a few hundred of these steps, each of which knew nothing about the ones still to come.\n\nThe chunks are called **tokens**. A token is roughly four characters of English, so about three quarters of a word. Common words are single tokens; unusual ones split. `understanding` is usually one token. `Rutherfordium` is four or five. A Hindi or Tamil sentence often costs two to three times as many tokens as the same sentence in English, which is why some tools feel slower and hit limits faster in those languages. Nobody chose that as policy. It falls out of how the vocabulary was built, mostly from English text.\n\nYou can see the split for yourself. OpenAI's tokenizer page and the free `tiktoken` Python library both show it; so does Hugging Face's tokenizer playground, which is free and needs no account.\n\n## Why it cannot count the letters in a word\n\nAsk a model how many times the letter r appears in *strawberry* and there is a decent chance of a wrong answer, even from a model that will happily discuss protein folding. This is not stupidity. The model never saw the letters. It saw the tokens `str`, `aw`, `berry`. Asking it to count characters is like asking you to count the pen strokes in a word somebody read aloud to you.\n\nThis matters more than it looks. The same blindness affects counting words in your essay, alphabetising a list, working with the digits of a long number, and any puzzle that turns on spelling. If you need those, use something that operates on characters: a word counter, a spreadsheet, `wc -w` in a terminal, or the model's code tool if it has one, which writes a Python program and runs it.\n\n## Where the knowledge sits\n\nThere is no database inside. During training, the model adjusted billions of numerical weights so that its next-token predictions matched an enormous quantity of text. What survives is a compressed statistical impression of that text. Facts repeated in thousands of places (the boiling point of water, the year the Second World War ended) are represented strongly and come out reliably. Facts stated once, in one obscure page, are represented weakly or not at all, and the model has no way to tell the difference between recalling one and constructing something that looks like one.\n\nThat is why it is excellent on the material in the first two chapters of your textbook and unreliable on the specific figure from the specific 2019 paper your lecturer mentioned.\n\n## Temperature, and why the same question gives different answers\n\nAt each step the model has a probability distribution over tokens. **Temperature** controls how much it favours the most likely one. At temperature 0 it takes the top token every time and the same prompt gives nearly the same answer. Higher temperature samples more widely and produces more variety.\n\nMost chat interfaces run somewhere in the middle, which is why asking the same question twice can give you two different answers, and occasionally one right and one wrong. This is worth exploiting: if you are unsure about an answer, open a fresh conversation and ask again in different words. Two independent agreeing answers are weak evidence. Two disagreeing answers are strong evidence that you need a real source.\n\n## What it does not have\n\n- **No memory between conversations**, unless a memory feature is switched on. A new chat starts blank.\n- **No sense of time.** Its training data stops at some date. It may not know that, and will often answer questions about last month as confidently as questions about last century.\n- **No access to your file** unless you paste it or upload it.\n- **No internal signal for truth.** It has a signal for *plausible continuation*. Those coincide often enough to be useful and diverge often enough to be dangerous.\n\n## The practical consequence\n\nEvery reliable technique in this course follows from this one mechanism. You ground it in text you supply, because supplied text is stronger than compressed memory. You ask for working, because a chain you can inspect is checkable and a conclusion is not. You verify anything specific, because specific things are the weakly-represented ones. You never let it do the counting.\n\nA model is a fluent, widely-read, extremely fast colleague with no memory, no source list, and no ability to tell you when it is guessing. Treat it exactly like that and most of the trouble disappears."},"check":{"question":"A student asks a model to check whether their 500-word limit essay is under the limit. It says 478 words. The real count is 611. What is the underlying reason?","options":[{"text":"The model was trained on older essays and has drifted, so its counting is out of date.","correct":false,"tempting":"Treats counting as a fact to be remembered rather than an operation to be performed; training date has nothing to do with arithmetic on the text in front of it."},{"text":"The essay exceeded the context window, so the model only saw part of it and counted what it could see.","correct":false,"tempting":"Plausible-sounding but wrong scale: a 611-word essay is roughly 800 tokens, far inside any modern context window."},{"text":"Temperature was set too high, so the model sampled a random number instead of the correct one.","correct":false,"tempting":"Confuses variability with capability; at temperature 0 the model would give a stable wrong count, not a right one."},{"text":"It reads text as tokens, not words, so it is estimating a plausible number rather than counting.","correct":true}]},"prev":{"slug":"what-happens-to-your-memory","title":"What Happens To Your Memory","url":"https://addaly.com/learn/ai-for-students/what-happens-to-your-memory"},"next":{"slug":"why-it-invents-things","title":"Why it invents things, and when","url":"https://addaly.com/learn/ai-for-students/why-it-invents-things"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What it is actually doing', in AI for Students. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-students/what-a-model-actually-does (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What it is actually doing","courseTitle":"AI for Students","module":{"index":1,"slug":"what-the-machine-is-doing","title":"What the machine is doing when it helps you"},"answer":null,"keyIdea":"A language model predicts the next token from a compressed statistical impression of its training text, so it is strong on repeated general material, weak on specific rare facts, and blind to the letters inside words.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"one-token-at-a-time","runs":[{"text":"One token at a time"}]},{"t":"p","runs":[{"text":"A language model does one thing. Given the text so far, it produces a probability for every possible next chunk of text, and one chunk is chosen. Then it does it again, with that chunk added to the text. Then again. An answer that took you fifteen seconds to read was built in a few hundred of these steps, each of which knew nothing about the ones still to come."}]},{"t":"p","runs":[{"text":"The chunks are called "},{"text":"tokens","b":true},{"text":". A token is roughly four characters of English, so about three quarters of a word. Common words are single tokens; unusual ones split. "},{"text":"understanding","code":true},{"text":" is usually one token. "},{"text":"Rutherfordium","code":true},{"text":" is four or five. A Hindi or Tamil sentence often costs two to three times as many tokens as the same sentence in English, which is why some tools feel slower and hit limits faster in those languages. Nobody chose that as policy. It falls out of how the vocabulary was built, mostly from English text."}]},{"t":"p","runs":[{"text":"You can see the split for yourself. OpenAI’s tokenizer page and the free "},{"text":"tiktoken","code":true},{"text":" Python library both show it; so does Hugging Face’s tokenizer playground, which is free and needs no account."}]},{"t":"h2","id":"why-it-cannot-count-the-letters-in-a-word","runs":[{"text":"Why it cannot count the letters in a word"}]},{"t":"p","runs":[{"text":"Ask a model how many times the letter r appears in "},{"text":"strawberry","i":true},{"text":" and there is a decent chance of a wrong answer, even from a model that will happily discuss protein folding. This is not stupidity. The model never saw the letters. It saw the tokens "},{"text":"str","code":true},{"text":", "},{"text":"aw","code":true},{"text":", "},{"text":"berry","code":true},{"text":". Asking it to count characters is like asking you to count the pen strokes in a word somebody read aloud to you."}]},{"t":"p","runs":[{"text":"This matters more than it looks. The same blindness affects counting words in your essay, alphabetising a list, working with the digits of a long number, and any puzzle that turns on spelling. If you need those, use something that operates on characters: a word counter, a spreadsheet, "},{"text":"wc -w","code":true},{"text":" in a terminal, or the model’s code tool if it has one, which writes a Python program and runs it."}]},{"t":"h2","id":"where-the-knowledge-sits","runs":[{"text":"Where the knowledge sits"}]},{"t":"p","runs":[{"text":"There is no database inside. During training, the model adjusted billions of numerical weights so that its next-token predictions matched an enormous quantity of text. What survives is a compressed statistical impression of that text. Facts repeated in thousands of places (the boiling point of water, the year the Second World War ended) are represented strongly and come out reliably. Facts stated once, in one obscure page, are represented weakly or not at all, and the model has no way to tell the difference between recalling one and constructing something that looks like one."}]},{"t":"p","runs":[{"text":"That is why it is excellent on the material in the first two chapters of your textbook and unreliable on the specific figure from the specific 2019 paper your lecturer mentioned."}]},{"t":"h2","id":"temperature-and-why-the-same-question-gives-different-answer","runs":[{"text":"Temperature, and why the same question gives different answers"}]},{"t":"p","runs":[{"text":"At each step the model has a probability distribution over tokens. "},{"text":"Temperature","b":true},{"text":" controls how much it favours the most likely one. At temperature 0 it takes the top token every time and the same prompt gives nearly the same answer. Higher temperature samples more widely and produces more variety."}]},{"t":"p","runs":[{"text":"Most chat interfaces run somewhere in the middle, which is why asking the same question twice can give you two different answers, and occasionally one right and one wrong. This is worth exploiting: if you are unsure about an answer, open a fresh conversation and ask again in different words. Two independent agreeing answers are weak evidence. Two disagreeing answers are strong evidence that you need a real source."}]},{"t":"h2","id":"what-it-does-not-have","runs":[{"text":"What it does not have"}]},{"t":"ul","items":[[{"text":"No memory between conversations","b":true},{"text":", unless a memory feature is switched on. A new chat starts blank."}],[{"text":"No sense of time.","b":true},{"text":" Its training data stops at some date. It may not know that, and will often answer questions about last month as confidently as questions about last century."}],[{"text":"No access to your file","b":true},{"text":" unless you paste it or upload it."}],[{"text":"No internal signal for truth.","b":true},{"text":" It has a signal for "},{"text":"plausible continuation","i":true},{"text":". Those coincide often enough to be useful and diverge often enough to be dangerous."}]]},{"t":"h2","id":"the-practical-consequence","runs":[{"text":"The practical consequence"}]},{"t":"p","runs":[{"text":"Every reliable technique in this course follows from this one mechanism. You ground it in text you supply, because supplied text is stronger than compressed memory. You ask for working, because a chain you can inspect is checkable and a conclusion is not. You verify anything specific, because specific things are the weakly-represented ones. You never let it do the counting."}]},{"t":"p","runs":[{"text":"A model is a fluent, widely-read, extremely fast colleague with no memory, no source list, and no ability to tell you when it is guessing. Treat it exactly like that and most of the trouble disappears."}]}],"check":{"question":[{"text":"A student asks a model to check whether their 500-word limit essay is under the limit. It says 478 words. The real count is 611. What is the underlying reason?"}],"options":[{"text":[{"text":"The model was trained on older essays and has drifted, so its counting is out of date."}],"correct":false,"tempting":[{"text":"Treats counting as a fact to be remembered rather than an operation to be performed; training date has nothing to do with arithmetic on the text in front of it."}]},{"text":[{"text":"The essay exceeded the context window, so the model only saw part of it and counted what it could see."}],"correct":false,"tempting":[{"text":"Plausible-sounding but wrong scale: a 611-word essay is roughly 800 tokens, far inside any modern context window."}]},{"text":[{"text":"Temperature was set too high, so the model sampled a random number instead of the correct one."}],"correct":false,"tempting":[{"text":"Confuses variability with capability; at temperature 0 the model would give a stable wrong count, not a right one."}]},{"text":[{"text":"It reads text as tokens, not words, so it is estimating a plausible number rather than counting."}],"correct":true,"tempting":null}],"order":[0,1,3,2],"why":null},"prev":{"slug":"what-happens-to-your-memory","title":"What Happens To Your Memory","minutes":8},"next":{"slug":"why-it-invents-things","title":"Why it invents things, and when","minutes":9}}}