{"course":{"slug":"maths-you-need","title":"The Maths You Actually Need","url":"https://addaly.com/learn/maths-you-need","level":"intermediate","track":"deep"},"lesson":{"slug":"probability-as-belief","title":"Probability is a degree of belief","url":"https://addaly.com/learn/maths-you-need/probability-as-belief","position":29,"of":76,"minutes":7,"answer":"A probability states a degree of belief; you judge it by calibration across many cases, never by one outcome.","keyIdea":"A probability states a degree of belief; you judge it by calibration across many cases, never by one outcome."},"body":{"format":"markdown","text":"## Two ways to read the same number\n\nSchool taught you the first one. Probability is a long-run frequency: roll a die 6000 times, expect roughly 1000 threes, so P(three) is 1/6. Something repeatable, counted.\n\nThat reading cannot handle most of what you care about. What is the probability that the rupee weakens against the dollar this quarter? That this patient has dengue rather than chikungunya? That the next word after \"I went to the\" is \"market\"? None of these repeat. There is one quarter, one patient, one sentence.\n\nThe second reading covers them. A probability is a stated degree of belief, given what you know. 0 means ruled out, 1 means certain, and everything real lives between. The number is not a property of the world. It is a property of your information about the world.\n\nThis is the reading a model uses, and once you have it, model outputs stop being mysterious.\n\n## What a model is actually saying\n\nA language model does not output words. It outputs a belief spread across every token in its vocabulary, conditioned on everything it has seen so far.\n\n```\nP(next = \"market\"  | \"I went to the\") = 0.21\nP(next = \"office\"  | \"I went to the\") = 0.14\nP(next = \"hospital\"| \"I went to the\") = 0.06\n... 50,000 more, all adding to 1.00\n```\n\nTwo rules constrain that list. Every number is at least zero, and they add to exactly one across the full set of options. Those two rules are why softmax exists: raw model scores can be negative or huge, and softmax exponentiates them (making everything positive) then divides by the total (making them sum to one). It converts scores into a statement of belief.\n\nThe vertical bar means \"given\". Everything a model reports is conditional. Change the context and the belief changes, which is why the same question with a different preamble gets a different answer. That is not the model being unstable. That is the model correctly having different beliefs given different information.\n\n## Calibration is the actual test\n\nIf a probability is a belief, how do you check it?\n\nNot on one case. If a doctor in Manila says 30% chance of dengue and it turns out to be dengue, she was not wrong. 30% things happen 30% of the time. That is what the number claims.\n\nYou check it in bulk. Collect every case where the doctor said 30%, and see whether about 3 in 10 turned out that way. Then do the same for her 70% cases and her 90% cases. If each bucket lands near its label, she is calibrated, and her numbers are worth something. If everything she calls 90% happens half the time, she is overconfident, and you should discount her.\n\nThat is the only honest test, and it applies to models exactly as written. Take 10,000 predictions the model gave 0.8 to, and count how many came true. Near 8,000 is calibrated. Near 5,000 means the model's confidence is decoration.\n\n## The uncomfortable part\n\nBase models tend to be reasonably calibrated. Instruction-tuned and RLHF-tuned models tend to be worse: the tuning rewards answers that sound helpful and assured, and hedging reads as unhelpful, so confidence gets inflated without accuracy following it up. A model that says \"I am 95% sure\" in prose is producing text that scored well with raters, which is a different thing from a belief you can bet on. The token probabilities underneath are more trustworthy than the sentence, though not by as much as you would like.\n\nSo: a confident model is not a correct model. When someone shows you a system that assigns probabilities, the question is never whether the last prediction was right. It is whether anyone has plotted the buckets."},"check":{"question":"A weather service said 90% chance of rain in Bogotá on Tuesday. Tuesday was dry. Was the forecast wrong?","options":[{"text":"Yes. 90% was a claim about that specific day, and the day was dry.","correct":false,"tempting":"Treats a high probability as a promise about a single event, so anything short of certainty is a broken promise."},{"text":"Yes, and it fits the well-documented pattern of overconfident forecasting systems.","correct":false,"tempting":"Uses a true general fact to justify a conclusion that one outcome cannot support."},{"text":"You cannot tell from one day. Gather every day it said 90% and check that rain came on roughly nine in ten.","correct":true},{"text":"The question does not apply. Tuesday happens once, so there is no long run and no way to score the forecast at all.","correct":false,"tempting":"Assumes a probability needs a repeatable event, so one-off forecasts are beyond judgement."}]},"prev":{"slug":"counting-before-probability","title":"Counting, and the size of the spaces involved","url":"https://addaly.com/learn/maths-you-need/counting-before-probability"},"next":{"slug":"conditional-probability","title":"Probability once you know something","url":"https://addaly.com/learn/maths-you-need/conditional-probability"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Probability is a degree of belief', in The Maths You Actually Need. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/maths-you-need/probability-as-belief (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Probability is a degree of belief","courseTitle":"The Maths You Actually Need","module":{"index":4,"slug":"probability-you-can-rely-on","title":"Probability you can rely on"},"answer":null,"keyIdea":"A probability states a degree of belief; you judge it by calibration across many cases, never by one outcome.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-ways-to-read-the-same-number","runs":[{"text":"Two ways to read the same number"}]},{"t":"p","runs":[{"text":"School taught you the first one. Probability is a long-run frequency: roll a die 6000 times, expect roughly 1000 threes, so P(three) is 1/6. Something repeatable, counted."}]},{"t":"p","runs":[{"text":"That reading cannot handle most of what you care about. What is the probability that the rupee weakens against the dollar this quarter? That this patient has dengue rather than chikungunya? That the next word after “I went to the” is “market”? None of these repeat. There is one quarter, one patient, one sentence."}]},{"t":"p","runs":[{"text":"The second reading covers them. A probability is a stated degree of belief, given what you know. 0 means ruled out, 1 means certain, and everything real lives between. The number is not a property of the world. It is a property of your information about the world."}]},{"t":"p","runs":[{"text":"This is the reading a model uses, and once you have it, model outputs stop being mysterious."}]},{"t":"h2","id":"what-a-model-is-actually-saying","runs":[{"text":"What a model is actually saying"}]},{"t":"p","runs":[{"text":"A language model does not output words. It outputs a belief spread across every token in its vocabulary, conditioned on everything it has seen so far."}]},{"t":"code","lang":"","label":"Text","text":"P(next = \"market\"  | \"I went to the\") = 0.21\nP(next = \"office\"  | \"I went to the\") = 0.14\nP(next = \"hospital\"| \"I went to the\") = 0.06\n... 50,000 more, all adding to 1.00"},{"t":"p","runs":[{"text":"Two rules constrain that list. Every number is at least zero, and they add to exactly one across the full set of options. Those two rules are why softmax exists: raw model scores can be negative or huge, and softmax exponentiates them (making everything positive) then divides by the total (making them sum to one). It converts scores into a statement of belief."}]},{"t":"p","runs":[{"text":"The vertical bar means “given”. Everything a model reports is conditional. Change the context and the belief changes, which is why the same question with a different preamble gets a different answer. That is not the model being unstable. That is the model correctly having different beliefs given different information."}]},{"t":"h2","id":"calibration-is-the-actual-test","runs":[{"text":"Calibration is the actual test"}]},{"t":"p","runs":[{"text":"If a probability is a belief, how do you check it?"}]},{"t":"p","runs":[{"text":"Not on one case. If a doctor in Manila says 30% chance of dengue and it turns out to be dengue, she was not wrong. 30% things happen 30% of the time. That is what the number claims."}]},{"t":"p","runs":[{"text":"You check it in bulk. Collect every case where the doctor said 30%, and see whether about 3 in 10 turned out that way. Then do the same for her 70% cases and her 90% cases. If each bucket lands near its label, she is calibrated, and her numbers are worth something. If everything she calls 90% happens half the time, she is overconfident, and you should discount her."}]},{"t":"p","runs":[{"text":"That is the only honest test, and it applies to models exactly as written. Take 10,000 predictions the model gave 0.8 to, and count how many came true. Near 8,000 is calibrated. Near 5,000 means the model’s confidence is decoration."}]},{"t":"h2","id":"the-uncomfortable-part","runs":[{"text":"The uncomfortable part"}]},{"t":"p","runs":[{"text":"Base models tend to be reasonably calibrated. Instruction-tuned and RLHF-tuned models tend to be worse: the tuning rewards answers that sound helpful and assured, and hedging reads as unhelpful, so confidence gets inflated without accuracy following it up. A model that says “I am 95% sure” in prose is producing text that scored well with raters, which is a different thing from a belief you can bet on. The token probabilities underneath are more trustworthy than the sentence, though not by as much as you would like."}]},{"t":"p","runs":[{"text":"So: a confident model is not a correct model. When someone shows you a system that assigns probabilities, the question is never whether the last prediction was right. It is whether anyone has plotted the buckets."}]}],"check":{"question":[{"text":"A weather service said 90% chance of rain in Bogotá on Tuesday. Tuesday was dry. Was the forecast wrong?"}],"options":[{"text":[{"text":"Yes. 90% was a claim about that specific day, and the day was dry."}],"correct":false,"tempting":[{"text":"Treats a high probability as a promise about a single event, so anything short of certainty is a broken promise."}]},{"text":[{"text":"Yes, and it fits the well-documented pattern of overconfident forecasting systems."}],"correct":false,"tempting":[{"text":"Uses a true general fact to justify a conclusion that one outcome cannot support."}]},{"text":[{"text":"You cannot tell from one day. Gather every day it said 90% and check that rain came on roughly nine in ten."}],"correct":true,"tempting":null},{"text":[{"text":"The question does not apply. Tuesday happens once, so there is no long run and no way to score the forecast at all."}],"correct":false,"tempting":[{"text":"Assumes a probability needs a repeatable event, so one-off forecasts are beyond judgement."}]}],"order":[1,0,3,2],"why":null},"prev":{"slug":"counting-before-probability","title":"Counting, and the size of the spaces involved","minutes":8},"next":{"slug":"conditional-probability","title":"Probability once you know something","minutes":8}}}