{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"pattern-or-understanding","title":"Does it understand anything? The honest answer","url":"https://addaly.com/learn/ai-actually-explained/pattern-or-understanding","position":8,"of":92,"minutes":9,"answer":"Whether a model understands anything is genuinely unsettled, so judge it by whether its failures look like the failures of something that understood: invented citations and reversals under pressure say no.","keyIdea":"Whether a model understands anything is genuinely unsettled, so judge it by whether its failures look like the failures of something that understood: invented citations and reversals under pressure say no."},"body":{"format":"markdown","text":"## The question people actually want answered\n\nEvery beginner arrives with this one, usually within the first hour, and it deserves a straight answer rather than a deflection. The straight answer is: **nobody knows, the experts disagree, and the disagreement is partly about the word.**\n\nThat is not a dodge. It is the state of the field in 2026, and anybody who tells you it is settled (in either direction) is telling you about their opinions.\n\nWhat follows is the shape of the argument, so you can follow it rather than pick a side.\n\n## The case that it is only pattern\n\nThe best-known version is the \"stochastic parrot\" argument, from a 2021 paper by Emily Bender, Timnit Gebru and colleagues. The claim: a language model is a system for stitching together sequences of language it has observed, according to probability, without any reference to meaning. It has never seen a dog, never been hungry, never had a purpose. Whatever it produces about the world is an artefact of how people write about the world.\n\nThe evidence offered is the failures. A system that understood would not confidently invent a court case that does not exist. It would not lose track of a simple physical situation four sentences in. It would not answer a puzzle correctly and then get the same puzzle wrong when a number is changed, which is what happens: change the digits in a maths word problem and accuracy drops measurably, which suggests the pattern was matched rather than the problem solved.\n\n## The case that something more is going on\n\nThe counter-argument is that the parrot picture predicts things that turn out to be false.\n\nA model that only recombined seen text should fail on new combinations. Models routinely handle them: a limerick about a specific plumbing fault in the style of a legal notice has never been written before.\n\nMore pointedly, researchers have gone looking inside. In one much-cited study, a model was trained purely on move sequences from the board game Othello, with no picture of a board anywhere in the data. Probing its internal activity revealed a representation of the board state, and altering that representation changed the model's next move in the way you would predict if it were using an internal board. Whatever that is, it is not a lookup table of sequences.\n\nOther work has found internal features that track things such as a text's language, its truth or falsity, or a geographic location, in ways that are used by the model's output rather than incidental.\n\n## Why the question is hard to settle\n\nBecause we have no test.\n\nEvery behavioural test proposed has been passed by systems that most people do not think understand anything, or is failed by people we know do. The Turing test fell decades ago to programs nobody claimed were intelligent. Exams get passed by models that then fail a rephrased question. \"Understanding\" in humans is not measured directly either; we infer it, from an assumption of similarity that does not transfer to a machine.\n\nUnderneath sits a genuine philosophical dispute about whether meaning can exist in a system with no contact with the world: an argument older than computing and no closer to resolution.\n\n## What to do instead\n\nYou do not have to settle it, and here is the good news: the practical question is different and answerable.\n\n**Does it fail in the ways something that understood would fail?**\n\nA person who has understood a document may misremember a detail, but will not invent a page number and a paragraph that were never in it. A person who has understood an argument may disagree with you, but will not reverse position because you sounded annoyed. A person who has understood a sum may make an error, but the error will usually be near-miss rather than a fluent, well-formatted answer with a wrong number in the middle.\n\nModels make the second kind of error routinely. That does not prove the parrot case, but it is the practical signature to watch for, and it tells you where to check.\n\n## The summary\n\nHold two things at once. The systems clearly do something more structured than word association: the internal-representation work makes the pure-parrot description hard to defend. And they clearly lack something a person has: the failures are not the failures of a mind that has understood.\n\nAnyone selling you certainty in either direction is selling. The useful stance is to treat \"does it understand\" as unresolved, and to make your decisions from observed failure patterns, which are much better documented than the metaphysics."},"check":{"question":"Why does the Othello study make the strict \"stochastic parrot\" description harder to defend?","options":[{"text":"Because the model beat expert human players, which shows a level of skill that memorisation could not reach.","correct":false,"tempting":"Substitutes performance for mechanism; strong play can come from memorised statistics and proves nothing about internal structure."},{"text":"Because the model was trained only on move sequences yet developed an internal representation of the board that, when altered, changed its next move.","correct":true},{"text":"Because the model could explain its moves in natural language, which requires an understanding of the rules.","correct":false,"tempting":"Treats fluent explanation as evidence; explanations are generated text and can be produced without any underlying representation."},{"text":"Because the researchers found the training data contained board diagrams the model had memorised.","correct":false,"tempting":"Reverses the finding: the point is that no board appeared in the data, so the representation was constructed rather than copied."}]},"prev":{"slug":"why-it-is-called-learning","title":"Why \"learning\" is a borrowed word","url":"https://addaly.com/learn/ai-actually-explained/why-it-is-called-learning"},"next":{"slug":"the-words-people-use-wrongly","title":"Five words that hide more than they say","url":"https://addaly.com/learn/ai-actually-explained/the-words-people-use-wrongly"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Does it understand anything? The honest answer', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/pattern-or-understanding (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Does it understand anything? The honest answer","courseTitle":"AI, Actually Explained","module":{"index":1,"slug":"what-the-word-names","title":"What this word actually names"},"answer":null,"keyIdea":"Whether a model understands anything is genuinely unsettled, so judge it by whether its failures look like the failures of something that understood: invented citations and reversals under pressure say no.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-question-people-actually-want-answered","runs":[{"text":"The question people actually want answered"}]},{"t":"p","runs":[{"text":"Every beginner arrives with this one, usually within the first hour, and it deserves a straight answer rather than a deflection. The straight answer is: "},{"text":"nobody knows, the experts disagree, and the disagreement is partly about the word.","b":true}]},{"t":"p","runs":[{"text":"That is not a dodge. It is the state of the field in 2026, and anybody who tells you it is settled (in either direction) is telling you about their opinions."}]},{"t":"p","runs":[{"text":"What follows is the shape of the argument, so you can follow it rather than pick a side."}]},{"t":"h2","id":"the-case-that-it-is-only-pattern","runs":[{"text":"The case that it is only pattern"}]},{"t":"p","runs":[{"text":"The best-known version is the “stochastic parrot” argument, from a 2021 paper by Emily Bender, Timnit Gebru and colleagues. The claim: a language model is a system for stitching together sequences of language it has observed, according to probability, without any reference to meaning. It has never seen a dog, never been hungry, never had a purpose. Whatever it produces about the world is an artefact of how people write about the world."}]},{"t":"p","runs":[{"text":"The evidence offered is the failures. A system that understood would not confidently invent a court case that does not exist. It would not lose track of a simple physical situation four sentences in. It would not answer a puzzle correctly and then get the same puzzle wrong when a number is changed, which is what happens: change the digits in a maths word problem and accuracy drops measurably, which suggests the pattern was matched rather than the problem solved."}]},{"t":"h2","id":"the-case-that-something-more-is-going-on","runs":[{"text":"The case that something more is going on"}]},{"t":"p","runs":[{"text":"The counter-argument is that the parrot picture predicts things that turn out to be false."}]},{"t":"p","runs":[{"text":"A model that only recombined seen text should fail on new combinations. Models routinely handle them: a limerick about a specific plumbing fault in the style of a legal notice has never been written before."}]},{"t":"p","runs":[{"text":"More pointedly, researchers have gone looking inside. In one much-cited study, a model was trained purely on move sequences from the board game Othello, with no picture of a board anywhere in the data. Probing its internal activity revealed a representation of the board state, and altering that representation changed the model’s next move in the way you would predict if it were using an internal board. Whatever that is, it is not a lookup table of sequences."}]},{"t":"p","runs":[{"text":"Other work has found internal features that track things such as a text’s language, its truth or falsity, or a geographic location, in ways that are used by the model’s output rather than incidental."}]},{"t":"h2","id":"why-the-question-is-hard-to-settle","runs":[{"text":"Why the question is hard to settle"}]},{"t":"p","runs":[{"text":"Because we have no test."}]},{"t":"p","runs":[{"text":"Every behavioural test proposed has been passed by systems that most people do not think understand anything, or is failed by people we know do. The Turing test fell decades ago to programs nobody claimed were intelligent. Exams get passed by models that then fail a rephrased question. “Understanding” in humans is not measured directly either; we infer it, from an assumption of similarity that does not transfer to a machine."}]},{"t":"p","runs":[{"text":"Underneath sits a genuine philosophical dispute about whether meaning can exist in a system with no contact with the world: an argument older than computing and no closer to resolution."}]},{"t":"h2","id":"what-to-do-instead","runs":[{"text":"What to do instead"}]},{"t":"p","runs":[{"text":"You do not have to settle it, and here is the good news: the practical question is different and answerable."}]},{"t":"p","runs":[{"text":"Does it fail in the ways something that understood would fail?","b":true}]},{"t":"p","runs":[{"text":"A person who has understood a document may misremember a detail, but will not invent a page number and a paragraph that were never in it. A person who has understood an argument may disagree with you, but will not reverse position because you sounded annoyed. A person who has understood a sum may make an error, but the error will usually be near-miss rather than a fluent, well-formatted answer with a wrong number in the middle."}]},{"t":"p","runs":[{"text":"Models make the second kind of error routinely. That does not prove the parrot case, but it is the practical signature to watch for, and it tells you where to check."}]},{"t":"h2","id":"the-summary","runs":[{"text":"The summary"}]},{"t":"p","runs":[{"text":"Hold two things at once. The systems clearly do something more structured than word association: the internal-representation work makes the pure-parrot description hard to defend. And they clearly lack something a person has: the failures are not the failures of a mind that has understood."}]},{"t":"p","runs":[{"text":"Anyone selling you certainty in either direction is selling. The useful stance is to treat “does it understand” as unresolved, and to make your decisions from observed failure patterns, which are much better documented than the metaphysics."}]}],"check":{"question":[{"text":"Why does the Othello study make the strict “stochastic parrot” description harder to defend?"}],"options":[{"text":[{"text":"Because the model beat expert human players, which shows a level of skill that memorisation could not reach."}],"correct":false,"tempting":[{"text":"Substitutes performance for mechanism; strong play can come from memorised statistics and proves nothing about internal structure."}]},{"text":[{"text":"Because the model was trained only on move sequences yet developed an internal representation of the board that, when altered, changed its next move."}],"correct":true,"tempting":null},{"text":[{"text":"Because the model could explain its moves in natural language, which requires an understanding of the rules."}],"correct":false,"tempting":[{"text":"Treats fluent explanation as evidence; explanations are generated text and can be produced without any underlying representation."}]},{"text":[{"text":"Because the researchers found the training data contained board diagrams the model had memorised."}],"correct":false,"tempting":[{"text":"Reverses the finding: the point is that no board appeared in the data, so the representation was constructed rather than copied."}]}],"order":[0,2,3,1],"why":null},"prev":{"slug":"why-it-is-called-learning","title":"Why “learning” is a borrowed word","minutes":8},"next":{"slug":"the-words-people-use-wrongly","title":"Five words that hide more than they say","minutes":8}}}