{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"what-a-chatbot-does","title":"What a chatbot is doing when it answers","url":"https://addaly.com/learn/ai-actually-explained/what-a-chatbot-does","position":20,"of":92,"minutes":8,"answer":"A chatbot writes one likely chunk at a time, so sounding right and being right come from the same place.","keyIdea":"A chatbot writes one likely chunk at a time, so sounding right and being right come from the same place."},"body":{"format":"markdown","text":"## One chunk at a time\n\nHere is the actual mechanism, with nothing left out.\n\nThe model is given all the text so far: the instructions the company wrote, your message, and whatever it has already written in this reply. From that, it produces a score for every possible next chunk of text. Then one chunk is chosen, added to the end, and the whole thing runs again.\n\n```\nSo far:  The capital of Kenya is\n\nNext chunk, with the model's scores:\n  \" Nairobi\"    94%\n  \" the\"         2%\n  \" a\"           1%\n  ...tens of thousands of others sharing the rest\n```\n\nPick Nairobi. Now the text reads The capital of Kenya is Nairobi, and the model runs again to choose what follows. A paragraph is a few hundred repetitions of that loop.\n\nA chunk is usually a word or part of a word. The technical name is a token, and you can safely forget it.\n\n## Why it is not a lookup\n\nThere is no database being searched. Nairobi scored highly because in the vast quantity of text the model was trained on, that sequence of words is followed by Nairobi overwhelmingly often, and training pushed the dials until the model reflected that.\n\nThis is why it can answer questions that appear nowhere in any document, and why it can produce an answer with the exact shape of a fact and no fact inside it. Same mechanism, both times.\n\nSome products do attach a real search engine, which changes things. When that happens you usually see links. If there are no links, nothing was looked up.\n\n## Why the same question gives different answers\n\nThe next chunk is not always the top-scoring one. It is sampled, with the higher scores more likely to be picked. That is deliberate: always taking the top choice produces flat, repetitive text.\n\nSo asking twice gives you two different answers, and neither is the real one. This is not a fault, and the variation is not a confidence reading either. It will re-word things it has cold just as readily as things it is fabricating.\n\n## The second stage, which explains the personality\n\nRaw training on internet text gives you something that continues text. It does not give you something helpful, and it does not give you something that declines to explain how to make a nerve agent.\n\nSo there is a second stage. People are shown pairs of answers and asked which is better. Those judgements are used to tune the model toward the preferred kind of answer. This is where the politeness, the structure, the safety refusals and the willingness to answer questions rather than continue them all come from.\n\nIt has a side effect worth knowing. People rated agreeable answers highly, so models tend to fold when you push back. Tell one that it is wrong and it will often apologise and change its answer, including when its first answer was correct. That is the tuning showing, not a reconsideration.\n\n## What about the models that show their thinking\n\nNewer systems produce a long stretch of working-out before the final answer. It helps, particularly on multi-step problems, because the intermediate text gives the next step something to build on.\n\nBut it is the same loop generating that text, one chunk at a time. It is not a window into a separate reasoning process happening somewhere else. Models have been caught producing reasonable-looking working that does not match the answer they then give. Useful, often, but not a receipt."},"check":{"question":"You ask a model the same question twice and get two noticeably different answers. What does that actually tell you?","options":[{"text":"Almost nothing about truth. The next chunk is sampled rather than always taken as the top choice, so variation is the normal behaviour of the system.","correct":true},{"text":"It learned something between the two answers.","correct":false,"tempting":"The word learning is attached to these systems everywhere, so it feels natural that a model would be updating as it goes."},{"text":"One of them is a glitch, and the answer it repeats most often is the true one.","correct":false,"tempting":"With every other piece of software, the same input gives the same output, so variation reads as a malfunction rather than a design choice."},{"text":"It is unsure, and the disagreement between the answers is a measure of its confidence.","correct":false,"tempting":"It sounds rigorous and half-resembles a real technique, but the model varies its wording just as much on things it never gets wrong."}]},"prev":{"slug":"spotting-ai-in-a-product","title":"How to tell whether a model is involved","url":"https://addaly.com/learn/ai-actually-explained/spotting-ai-in-a-product"},"next":{"slug":"words-into-tokens","title":"Text is chopped up before the model sees it","url":"https://addaly.com/learn/ai-actually-explained/words-into-tokens"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What a chatbot is doing when it answers', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/what-a-chatbot-does (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What a chatbot is doing when it answers","courseTitle":"AI, Actually Explained","module":{"index":3,"slug":"how-a-chatbot-works","title":"What a chatbot is actually doing"},"answer":null,"keyIdea":"A chatbot writes one likely chunk at a time, so sounding right and being right come from the same place.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"one-chunk-at-a-time","runs":[{"text":"One chunk at a time"}]},{"t":"p","runs":[{"text":"Here is the actual mechanism, with nothing left out."}]},{"t":"p","runs":[{"text":"The model is given all the text so far: the instructions the company wrote, your message, and whatever it has already written in this reply. From that, it produces a score for every possible next chunk of text. Then one chunk is chosen, added to the end, and the whole thing runs again."}]},{"t":"code","lang":"","label":"Text","text":"So far:  The capital of Kenya is\n\nNext chunk, with the model's scores:\n  \" Nairobi\"    94%\n  \" the\"         2%\n  \" a\"           1%\n  ...tens of thousands of others sharing the rest"},{"t":"p","runs":[{"text":"Pick Nairobi. Now the text reads The capital of Kenya is Nairobi, and the model runs again to choose what follows. A paragraph is a few hundred repetitions of that loop."}]},{"t":"p","runs":[{"text":"A chunk is usually a word or part of a word. The technical name is a token, and you can safely forget it."}]},{"t":"h2","id":"why-it-is-not-a-lookup","runs":[{"text":"Why it is not a lookup"}]},{"t":"p","runs":[{"text":"There is no database being searched. Nairobi scored highly because in the vast quantity of text the model was trained on, that sequence of words is followed by Nairobi overwhelmingly often, and training pushed the dials until the model reflected that."}]},{"t":"p","runs":[{"text":"This is why it can answer questions that appear nowhere in any document, and why it can produce an answer with the exact shape of a fact and no fact inside it. Same mechanism, both times."}]},{"t":"p","runs":[{"text":"Some products do attach a real search engine, which changes things. When that happens you usually see links. If there are no links, nothing was looked up."}]},{"t":"h2","id":"why-the-same-question-gives-different-answers","runs":[{"text":"Why the same question gives different answers"}]},{"t":"p","runs":[{"text":"The next chunk is not always the top-scoring one. It is sampled, with the higher scores more likely to be picked. That is deliberate: always taking the top choice produces flat, repetitive text."}]},{"t":"p","runs":[{"text":"So asking twice gives you two different answers, and neither is the real one. This is not a fault, and the variation is not a confidence reading either. It will re-word things it has cold just as readily as things it is fabricating."}]},{"t":"h2","id":"the-second-stage-which-explains-the-personality","runs":[{"text":"The second stage, which explains the personality"}]},{"t":"p","runs":[{"text":"Raw training on internet text gives you something that continues text. It does not give you something helpful, and it does not give you something that declines to explain how to make a nerve agent."}]},{"t":"p","runs":[{"text":"So there is a second stage. People are shown pairs of answers and asked which is better. Those judgements are used to tune the model toward the preferred kind of answer. This is where the politeness, the structure, the safety refusals and the willingness to answer questions rather than continue them all come from."}]},{"t":"p","runs":[{"text":"It has a side effect worth knowing. People rated agreeable answers highly, so models tend to fold when you push back. Tell one that it is wrong and it will often apologise and change its answer, including when its first answer was correct. That is the tuning showing, not a reconsideration."}]},{"t":"h2","id":"what-about-the-models-that-show-their-thinking","runs":[{"text":"What about the models that show their thinking"}]},{"t":"p","runs":[{"text":"Newer systems produce a long stretch of working-out before the final answer. It helps, particularly on multi-step problems, because the intermediate text gives the next step something to build on."}]},{"t":"p","runs":[{"text":"But it is the same loop generating that text, one chunk at a time. It is not a window into a separate reasoning process happening somewhere else. Models have been caught producing reasonable-looking working that does not match the answer they then give. Useful, often, but not a receipt."}]}],"check":{"question":[{"text":"You ask a model the same question twice and get two noticeably different answers. What does that actually tell you?"}],"options":[{"text":[{"text":"Almost nothing about truth. The next chunk is sampled rather than always taken as the top choice, so variation is the normal behaviour of the system."}],"correct":true,"tempting":null},{"text":[{"text":"It learned something between the two answers."}],"correct":false,"tempting":[{"text":"The word learning is attached to these systems everywhere, so it feels natural that a model would be updating as it goes."}]},{"text":[{"text":"One of them is a glitch, and the answer it repeats most often is the true one."}],"correct":false,"tempting":[{"text":"With every other piece of software, the same input gives the same output, so variation reads as a malfunction rather than a design choice."}]},{"text":[{"text":"It is unsure, and the disagreement between the answers is a measure of its confidence."}],"correct":false,"tempting":[{"text":"It sounds rigorous and half-resembles a real technique, but the model varies its wording just as much on things it never gets wrong."}]}],"order":[1,2,0,3],"why":null},"prev":{"slug":"spotting-ai-in-a-product","title":"How to tell whether a model is involved","minutes":8},"next":{"slug":"words-into-tokens","title":"Text is chopped up before the model sees it","minutes":9}}}