{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"counting-and-arithmetic","title":"Why it cannot count, and what to do instead","url":"https://addaly.com/learn/ai-actually-explained/counting-and-arithmetic","position":34,"of":92,"minutes":8,"answer":"Counting fails because the model sees tokens rather than letters and arithmetic fails because it predicts rather than computes, so hand the deterministic part to a program and keep the judgement for the model.","keyIdea":"Counting fails because the model sees tokens rather than letters and arithmetic fails because it predicts rather than computes, so hand the deterministic part to a program and keep the judgement for the model."},"body":{"format":"markdown","text":"## Two separate problems\n\nPeople lump these together and they have different causes.\n\n**Counting characters** fails because of tokenisation. The model never sees letters. Asked how many times `r` appears in \"strawberry\", it must reason about spelling from what it has read about spelling, rather than by inspecting the word. Sometimes that works. It is not a look-up.\n\nThe same applies to reversing a word, counting words in a paragraph, producing exactly 100 words, or finding all words in a text starting with a given letter. Each asks for an operation on units the model does not perceive.\n\n**Arithmetic** fails for a different reason. There is no calculator inside. Multiplication is not performed; it is predicted. And the digits of a long number are split into tokens by frequency, not by place value, so 4,871,203 might arrive as three chunks that do not align with hundreds, thousands and millions. Column arithmetic, the method every human uses, is unavailable.\n\nThe practical shape: single-digit and small sums are usually right, because they appeared constantly in the training text. Multiplying two five-digit numbers is frequently wrong, and wrong in the confident, well-formatted way. Percentages of large numbers, compound interest over many periods and unit conversions with awkward factors all sit in the danger zone.\n\n## Why the errors are so plausible\n\nA person doing long multiplication badly produces an obviously mangled answer. A model produces an answer of the right magnitude, the right number of digits, and the right shape: wrong in the middle.\n\nThat is worse than an obvious error, because it survives a glance. If you are reviewing a document containing model-produced figures, the sanity check most people apply (does this look about right) is precisely the check this failure mode passes.\n\n## The fix, which is not a better prompt\n\nGive the arithmetic to something that does arithmetic.\n\n**Ask for code, then run it.** \"Write Python that computes this\" produces a short, readable program. Python does the sum exactly. You can check the program's logic, which is a far easier review task than checking a number. Python is free, and if you have no installation, a free Google Colab notebook runs in a browser on a phone.\n\n**Use a tool-enabled assistant.** Most major products now execute code or call a calculator when a question needs it. When the interface shows that a tool ran, the arithmetic is genuine. When it does not, it is predicted. Look for the difference: it is usually visible in the response.\n\n**Use a spreadsheet for anything with repeated structure.** LibreOffice Calc is free and exact. Ask the model for the formula, not for the answer.\n\nThe underlying principle is worth stating on its own, because it generalises well beyond arithmetic: **give the deterministic part to a program and the judgement part to the model.** Deciding which figures matter, what the comparison should be, and how to present it: that is where a model helps. Computing the number is not.\n\n> Figure: Split the task: deterministic to a program, judgement to the model. Give it to a program: Counting letters, words or characters; Multiplying, percentages, compound interest; Unit conversions with awkward factors; Anything that must total exactly; Free path: Python, a Colab notebook on a phone, or LibreOffice Calc. Keep it for the model: Deciding which figures matter; Choosing the comparison to make; Writing the formula or the code that does the sum; Explaining the result to a reader; Spotting that a table's parts do not add up to its total. The model sees tokens, not letters, and predicts rather than computes, so it produces answers of the right magnitude and the right number of digits, wrong in the middle. That is worse than an obvious error, because it survives a glance.\n\n## Where reasoning models change the picture, and where they do not\n\nModels trained to work through problems step by step before answering are substantially better at mathematics. They score highly on competition problems that older models failed outright. This is real progress and it is not marketing.\n\nBut two cautions. First, they are still predicting tokens; the improvement comes from doing the working in text where each step is short and well-supported, not from acquiring an arithmetic unit. Long multiplication of arbitrary large numbers remains shaky. Second, they cost more and take longer (often ten to sixty seconds) which is poor value for a task a spreadsheet does exactly and instantly.\n\n## A checklist for numbers in any model output\n\n- Did a tool run, or was the number generated? If you cannot tell, assume generated.\n- Is the magnitude plausible against something you know independently?\n- Do the parts sum to the total? Model-produced tables of figures frequently do not, and this is the fastest single check available.\n- Are percentages of a stated base actually that percentage of that base?\n- Where a figure will be acted on, recompute it yourself once.\n\nThat last one sounds tedious and takes under a minute. It has caught errors in board papers, invoices and dosage calculations, and each of those has appeared in reporting since 2023."},"check":{"question":"A model is asked to total twelve invoice lines and returns a figure of the right magnitude that is wrong by a few hundred. Why is this error more dangerous than an obviously mangled one?","options":[{"text":"Because it indicates the model misread the input figures, which suggests all twelve lines are unreliable.","correct":false,"tempting":"Assumes a reading failure; the figures were usually read correctly and the summation was predicted rather than performed."},{"text":"Because the model will repeat the same wrong figure consistently, making it look verified.","correct":false,"tempting":"Sampling usually varies the error between runs, so consistency is not what makes this dangerous."},{"text":"Because it passes the only check most reviewers apply (whether the total looks about right) while being wrong in a way that only recomputation reveals.","correct":true},{"text":"Because arithmetic errors compound in later calculations, whereas obvious errors do not.","correct":false,"tempting":"True of any error carried forward, and does not explain why plausibility specifically defeats review."}]},"prev":{"slug":"what-it-cannot-do","title":"What it cannot do, honestly","url":"https://addaly.com/learn/ai-actually-explained/what-it-cannot-do"},"next":{"slug":"the-cut-off-and-the-clock","title":"It has no clock, and its knowledge thins before it stops","url":"https://addaly.com/learn/ai-actually-explained/the-cut-off-and-the-clock"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Why it cannot count, and what to do instead', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/counting-and-arithmetic (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Why it cannot count, and what to do instead","courseTitle":"AI, Actually Explained","module":{"index":4,"slug":"why-it-goes-wrong","title":"Why it gets things wrong"},"answer":null,"keyIdea":"Counting fails because the model sees tokens rather than letters and arithmetic fails because it predicts rather than computes, so hand the deterministic part to a program and keep the judgement for the model.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"left":{"title":"Give it to a program","items":["Counting letters, words or characters","Multiplying, percentages, compound interest","Unit conversions with awkward factors","Anything that must total exactly","Free path: Python, a Colab notebook on a phone, or LibreOffice Calc"]},"type":"compare","right":{"title":"Keep it for the model","items":["Deciding which figures matter","Choosing the comparison to make","Writing the formula or the code that does the sum","Explaining the result to a reader","Spotting that a table’s parts do not add up to its total"]},"title":"Split the task: deterministic to a program, judgement to the model","caption":"The model sees tokens, not letters, and predicts rather than computes, so it produces answers of the right magnitude and the right number of digits, wrong in the middle. That is worse than an obvious error, because it survives a glance."},"alt":"Figure: Split the task: deterministic to a program, judgement to the model. Give it to a program: Counting letters, words or characters; Multiplying, percentages, compound interest; Unit conversions with awkward factors; Anything that must total exactly; Free path: Python, a Colab notebook on a phone, or LibreOffice Calc. Keep it for the model: Deciding which figures matter; Choosing the comparison to make; Writing the formula or the code that does the sum; Explaining the result to a reader; Spotting that a table’s parts do not add up to its total. The model sees tokens, not letters, and predicts rather than computes, so it produces answers of the right magnitude and the right number of digits, wrong in the middle. That is worse than an obvious error, because it survives a glance."}},"blocks":[{"t":"h2","id":"two-separate-problems","runs":[{"text":"Two separate problems"}]},{"t":"p","runs":[{"text":"People lump these together and they have different causes."}]},{"t":"p","runs":[{"text":"Counting characters","b":true},{"text":" fails because of tokenisation. The model never sees letters. Asked how many times "},{"text":"r","code":true},{"text":" appears in “strawberry”, it must reason about spelling from what it has read about spelling, rather than by inspecting the word. Sometimes that works. It is not a look-up."}]},{"t":"p","runs":[{"text":"The same applies to reversing a word, counting words in a paragraph, producing exactly 100 words, or finding all words in a text starting with a given letter. Each asks for an operation on units the model does not perceive."}]},{"t":"p","runs":[{"text":"Arithmetic","b":true},{"text":" fails for a different reason. There is no calculator inside. Multiplication is not performed; it is predicted. And the digits of a long number are split into tokens by frequency, not by place value, so 4,871,203 might arrive as three chunks that do not align with hundreds, thousands and millions. Column arithmetic, the method every human uses, is unavailable."}]},{"t":"p","runs":[{"text":"The practical shape: single-digit and small sums are usually right, because they appeared constantly in the training text. Multiplying two five-digit numbers is frequently wrong, and wrong in the confident, well-formatted way. Percentages of large numbers, compound interest over many periods and unit conversions with awkward factors all sit in the danger zone."}]},{"t":"h2","id":"why-the-errors-are-so-plausible","runs":[{"text":"Why the errors are so plausible"}]},{"t":"p","runs":[{"text":"A person doing long multiplication badly produces an obviously mangled answer. A model produces an answer of the right magnitude, the right number of digits, and the right shape: wrong in the middle."}]},{"t":"p","runs":[{"text":"That is worse than an obvious error, because it survives a glance. If you are reviewing a document containing model-produced figures, the sanity check most people apply (does this look about right) is precisely the check this failure mode passes."}]},{"t":"h2","id":"the-fix-which-is-not-a-better-prompt","runs":[{"text":"The fix, which is not a better prompt"}]},{"t":"p","runs":[{"text":"Give the arithmetic to something that does arithmetic."}]},{"t":"p","runs":[{"text":"Ask for code, then run it.","b":true},{"text":" “Write Python that computes this” produces a short, readable program. Python does the sum exactly. You can check the program’s logic, which is a far easier review task than checking a number. Python is free, and if you have no installation, a free Google Colab notebook runs in a browser on a phone."}]},{"t":"p","runs":[{"text":"Use a tool-enabled assistant.","b":true},{"text":" Most major products now execute code or call a calculator when a question needs it. When the interface shows that a tool ran, the arithmetic is genuine. When it does not, it is predicted. Look for the difference: it is usually visible in the response."}]},{"t":"p","runs":[{"text":"Use a spreadsheet for anything with repeated structure.","b":true},{"text":" LibreOffice Calc is free and exact. Ask the model for the formula, not for the answer."}]},{"t":"p","runs":[{"text":"The underlying principle is worth stating on its own, because it generalises well beyond arithmetic: "},{"text":"give the deterministic part to a program and the judgement part to the model.","b":true},{"text":" Deciding which figures matter, what the comparison should be, and how to present it: that is where a model helps. Computing the number is not."}]},{"t":"h2","id":"where-reasoning-models-change-the-picture-and-where-they-do-","runs":[{"text":"Where reasoning models change the picture, and where they do not"}]},{"t":"p","runs":[{"text":"Models trained to work through problems step by step before answering are substantially better at mathematics. They score highly on competition problems that older models failed outright. This is real progress and it is not marketing."}]},{"t":"p","runs":[{"text":"But two cautions. First, they are still predicting tokens; the improvement comes from doing the working in text where each step is short and well-supported, not from acquiring an arithmetic unit. Long multiplication of arbitrary large numbers remains shaky. Second, they cost more and take longer (often ten to sixty seconds) which is poor value for a task a spreadsheet does exactly and instantly."}]},{"t":"h2","id":"a-checklist-for-numbers-in-any-model-output","runs":[{"text":"A checklist for numbers in any model output"}]},{"t":"ul","items":[[{"text":"Did a tool run, or was the number generated? If you cannot tell, assume generated."}],[{"text":"Is the magnitude plausible against something you know independently?"}],[{"text":"Do the parts sum to the total? Model-produced tables of figures frequently do not, and this is the fastest single check available."}],[{"text":"Are percentages of a stated base actually that percentage of that base?"}],[{"text":"Where a figure will be acted on, recompute it yourself once."}]]},{"t":"p","runs":[{"text":"That last one sounds tedious and takes under a minute. It has caught errors in board papers, invoices and dosage calculations, and each of those has appeared in reporting since 2023."}]}],"check":{"question":[{"text":"A model is asked to total twelve invoice lines and returns a figure of the right magnitude that is wrong by a few hundred. Why is this error more dangerous than an obviously mangled one?"}],"options":[{"text":[{"text":"Because it indicates the model misread the input figures, which suggests all twelve lines are unreliable."}],"correct":false,"tempting":[{"text":"Assumes a reading failure; the figures were usually read correctly and the summation was predicted rather than performed."}]},{"text":[{"text":"Because the model will repeat the same wrong figure consistently, making it look verified."}],"correct":false,"tempting":[{"text":"Sampling usually varies the error between runs, so consistency is not what makes this dangerous."}]},{"text":[{"text":"Because it passes the only check most reviewers apply (whether the total looks about right) while being wrong in a way that only recomputation reveals."}],"correct":true,"tempting":null},{"text":[{"text":"Because arithmetic errors compound in later calculations, whereas obvious errors do not."}],"correct":false,"tempting":[{"text":"True of any error carried forward, and does not explain why plausibility specifically defeats review."}]}],"order":[0,2,3,1],"why":null},"prev":{"slug":"what-it-cannot-do","title":"What it cannot do, honestly","minutes":8},"next":{"slug":"the-cut-off-and-the-clock","title":"It has no clock, and its knowledge thins before it stops","minutes":8}}}