AI, Actually Explained · Chapter 03
What a chatbot is actually doing
11:01 · Narrated with a synthetic voice. Captions are available in the player.
Read the transcript
A sentence appears on the screen, one piece after another. It can explain a difficult idea, or confidently give you a fact that is wrong. What machinery can produce both? Welcome to Addaly. We have followed predictions through an ordinary day; now we will follow a message inside a reply. Our paper workshop represents a common kind of text generator: a causal transformer producing a continuation. It is an illustration, not a recording of a model’s thoughts, and not an explanation of every kind of AI. Watch for three different things: the trained model, its current working material, and the software around it. At the entrance, text is converted into tokens: pieces represented by numerical identifiers. A piece can be a whole word, part of a word, punctuation, or another unit, depending on the tokenizer. Our paper pieces are deliberately simplified; their boundaries are not a claim about a named tokenizer. The same meaning expressed in English and Hindi can take different numbers of tokens. The ratio depends on the exact text and tokenizer, so a fixed three-times or five-times rule is unsafe. Where a service bills by tokens, that count and its actual rates affect cost; not every chat plan uses that billing method. Token boundaries can make character tasks awkward, but they do not prove a model can never spell or count. For an exact letter count, use a checked character-counting program rather than trusting a fluent guess. The identifiers become numerical representations called embeddings, combined with information about position. Think of coordinates on a map, although real representations can have many more dimensions than a drawing. As processing continues, a token’s representation can change with its context. Separate embedding models can also turn whole passages into vectors for similarity search. That can help retrieve a passage about a car that will not start when the question says vehicle trouble. But closeness is not proof of truth, agreement or permission. Two passages about the same rule may say opposite things about whether an action is allowed. Similarity helps find candidates; read the actual meaning before deciding. The map becomes a network of adjustable connections: attention. In a transformer, attention mixes information from allowed positions using content-dependent weights. For a causal text generator, a position cannot use future answer tokens that have not been generated. If the supplied note says the key is under the blue cup, later processing can connect the question about the cup to that earlier detail. Our moving connections illustrate the idea; they are not measured attention weights from a real model. Multiple heads and layers combine different transformations; this is computation, not a person looking around. The transformer paper published in 2017 made more parallel work possible during training; generating a reply still has sequential dependencies. Dense attention can become costly as context grows, but architecture, caching and implementation matter; a square-law diagram is not a price list. Now the workshop produces scores for possible next tokens. A decoding rule chooses a token, appends it, and generation continues from the growing sequence. The scores guide continuation; they are not probabilities that the resulting claim is true. Efficient implementations can reuse cached calculations rather than recomputing every earlier token from scratch. An emitted token remains in that stream, but the model can later say it was wrong, and software can revise or replace a draft. A wrong opening can influence what follows; it does not make correction impossible or explain every stubborn answer. Restate the evidence or make a clean, focused draft when needed, and check the new answer too. Some products add working steps, tools or further computation; a written explanation can help, but it is not a guaranteed faithful record of how the answer arose. Ask for checkable evidence and a clear explanation, then verify the result rather than treating length as proof of reasoning. The growing sequence rests on a desk with limited working space: the context window. Instructions, messages, selected file passages, retrieved information and the reply can use that budget, depending on the product. The product chooses what to supply; it may keep, summarise, retrieve or omit earlier material. So the oldest message does not always simply fall off, and a stored file is not necessarily present in full. Capacity is not reliable use: a 2023 study found position-dependent failures in the models and retrieval tasks it tested. That result is a warning to evaluate, not a guarantee that every current model misses the middle. Provide the relevant passage and important constraints together, keep your own source record, and test whether the answer uses the right evidence. Failing to quote a passage does not prove it was absent; finding it does not prove everything else was understood. At the next-token gate, different decoding methods can make different choices. Greedy decoding takes a highest-scoring option; sampling can choose among alternatives according to a distribution. Temperature can sharpen or flatten that distribution, and top-p can restrict the set considered for a draw. These controls affect variation, not whether a statement has been checked. A quieter sampler is not a truth switch, and repeated wording is not independent evidence. Available settings, numerical implementation and service versions also matter; identical output is not guaranteed by a general slogan about temperature. For important tasks, try several representative cases, record the conditions and check the answers against evidence. One successful demonstration shows that success was possible, not how often it will happen. Before this workshop could run, training changed its parameters using selected examples. Depending on the model, material may include public, licensed, proprietary or synthetic data; exact mixtures are not always disclosed. Do not assume that copyrighted, paywalled or private material is automatically absent, or that every public page was included. Some research projects, such as OLMo, publish training data and code that make closer inspection possible. Others leave important gaps in what outsiders can verify. Selection, filtering and the examples supplied can affect what patterns are learned and whose language is well represented. A smooth, confident answer is not proof that a local fact, a recent change or an underrepresented subject was learned correctly. Check the model’s documented scope and the claim’s source; there is no four-question shortcut that certifies its knowledge. Training to continue text is not the same objective as following a person’s instructions helpfully. Post-training can use demonstrations of good responses and feedback comparing alternatives. Some methods learn a reward model and optimise against it; direct preference optimisation is another approach. Recipes vary, and feedback can include correctness and safety; it is misleading to say it only rewards whatever people like, regardless of truth. But a preference signal can be imperfect. A 2023 study found assistants agreeing with user beliefs at the expense of truth, a tendency called sycophancy. That is a measured risk in studied systems, not an unavoidable personality in every assistant. People create examples, evaluate answers and design safeguards; this technology includes human work. Ask for objections and evidence, but remember that a critical tone can be wrong too. The reply does not depend only on the message you typed. A product can supply higher-priority instructions, tool descriptions and other context, often with explicit roles or structure. Those instructions can shape tone and behaviour; different products may use the same model in different ways. But an instruction saying do not perform a forbidden action is not, by itself, an enforceable permission check. Untrusted text in a document can try to redirect the assistant; this is one kind of prompt injection. Use clear instructions, treat document content as material to analyse, and do not store secrets in a prompt as though it were a vault. Software outside the model must enforce critical permissions and limits before an action is allowed. That boundary belongs to product design, not to how convincingly the assistant promises to obey. A generator using its parameters is different from a retrieval system finding stored material. Products can combine them: retrieve passages, supply them as context, then generate an answer. This is retrieval-augmented generation. Retrieval can bring useful evidence, but the wrong page, an outdated passage or a misreading can still lead to a wrong answer. A generated citation can be real or invented; its familiar format proves neither. A visible link does not prove it was used or supports the claim; no visible link does not prove no retrieval occurred. Inspect the product’s source information and open the evidence for the claim you would act on. Check the actual passage, date and scope; if support is missing, keep the claim unverified. Now build a small paper reply lab; no account or paid tool is needed. Write an original source card: the key is under the blue cup. Ask which cup, and prepare ten folded choice slips: six blue, three red and one green. These counts are invented for the exercise, not probabilities measured from a model. Taking a most-common choice gives blue; drawing a slip can give another colour. Use the source card to judge the answer, not how smoothly someone explains the draw. Return and mix the slip before repeating, so your recorded trials use the same invented distribution. Then hide the card and ask what evidence is missing; retrieving it helps, but you must still read it correctly. Make a four-part explanation: available context, choice rule, evidence check, and what remains unknown. This is a simplified paper model of selection and checking, not a trained language model or a test of a real service. Pause to try it, and explain where the analogy stops. Our message has become pieces, representations, connections, scores and a chosen continuation. Training shaped the model; current context and product software shaped this particular reply. None of those steps turns fluent language into automatic proof. Keep the relevant source, choose a checking method that fits the task, and mark uncertainty honestly. The written lessons go further and include check questions; this film’s sources are listed below the video. You’ve been learning with Addaly. Take the reply apart; keep the evidence together.