Addaly is in open beta. Things will change, and AI answers can be wrong — check anything that matters.

The Maths You Actually Need

Eight ideas that carry almost all the weight in machine learning.

Lesson 35 of 7610 min

Six distributions, and the process each one describes

Pick by the process, not by the shape

The usual way distributions are taught is as a gallery of curves. That is backwards. Each one is the answer to a specific question about how something is generated, and if you know the process you know the distribution. Fitting curves by eye and choosing the best-looking one is how people end up modelling counts with a normal and being surprised by negative predictions.

Six cover almost everything you will meet.

Bernoulli: one yes-or-no event

One trial, probability p of success. Mean p, variance p(1 − p).

Every binary label is a Bernoulli. Every click, conversion, correct answer, spam flag. The variance formula is worth noticing: it is largest at p = 0.5, where it is 0.25, and falls to zero at both extremes. A coin is maximally uncertain; an event that almost never happens is nearly predictable.

Binomial: how many successes in n trials

n independent Bernoulli trials with the same p. Mean np, variance np(1 − p).

This is the distribution of your accuracy count on a test set. Score 87 out of 100 and the observed proportion is 0.87 with a standard error of

sqrt(0.87 * 0.13 / 100) = 0.034

That is 3.4 percentage points, which is why an 87 per cent and an 84 per cent on a 100-item test set are not distinguishable. The binomial is where the honest error bar on every accuracy figure comes from.

Its assumption is that trials are independent and share the same p. Test items from the same document break the first; a test set mixing easy and hard cases breaks the second, in the direction of making the true variance larger than the formula says.

Categorical: one draw from k options

The generalisation of Bernoulli to k outcomes with probabilities summing to 1. This is precisely what a language model's softmax output is: a categorical distribution over the vocabulary, from which one token is drawn. Nothing more exotic is happening at the sampling step.

Poisson: counts of rare events in a window

Events happening independently at a constant average rate λ per interval. Mean λ, and — the distinguishing feature — variance also λ.

Support tickets per hour, errors per thousand requests, arrivals per minute. The mean-equals-variance property gives you a free diagnostic: compute both on your count data. If the variance is much larger than the mean, the events are not independent — they arrive in bursts — and the Poisson assumption is wrong. This is called overdispersion, it is extremely common in real systems, and the standard replacement is the negative binomial.

Ignoring overdispersion produces confidence intervals that are far too narrow, which is how monitoring systems end up with alert thresholds that fire constantly.

Exponential: waiting time between Poisson events

If events are Poisson with rate λ, the gap between consecutive events is exponential with mean 1/λ.

It has one strange and consequential property: it is memoryless. If you have waited 10 minutes for the next event, the distribution of the remaining wait is identical to what it was at the start. Nothing accumulates.

This is why the exponential is a reasonable model for the gaps between independent arrivals and a poor model for anything with wear, ageing or a schedule. Time until a machine fails is not memoryless; the machine gets older.

Log-normal: the product of many small effects

If a quantity is the product of many independent factors rather than their sum, its logarithm is normal and the quantity itself is log-normal. Right-skewed, always positive, with a long upper tail.

This describes an enormous share of real measurements: incomes, file sizes, city populations, request latencies, session durations, revenue per customer. The normal distribution arises from adding many small effects; the log-normal arises from multiplying them, and multiplication is at least as common in the world.

The practical signal that you are looking at one: the mean is well above the median, and taking logs makes the histogram symmetric. When that happens, model the log. A regression on log-revenue is usually better behaved in every way than a regression on revenue.

Name the process, and the distribution followsHow the number was generatedOne yes-or-no eventA count of successes in n independent triesOne draw from k optionsRare events at a steady rate in a fixed windowThe gap between independent events, nothingageingMany independent factors multiplied togetherThe distribution that describes itBernoulli: mean p, variance p times one minuspBinomial: where an accuracy's error bar comesfromCategorical: a language model's softmax outputPoisson: check that the variance equals themeanExponential: memoryless, so waiting resetsnothingLog-normal: right-skewed, mean well abovemedianChoosing a distribution is a claim about how the data were produced, not a question of which curvefits best. When counts show a variance far above their mean, the Poisson story is wrong, and everyinterval built on it is far too narrow.
Name the process, and the distributionfollowsHow the number was generatedOne yes-or-no eventA count of successes in n independent triesOne draw from k optionsRare events at a steady rate in a fixedwindowThe gap between independent events, nothingageingMany independent factors multiplied togetherThe distribution that describes itBernoulli: mean p, variance p times oneminus pBinomial: where an accuracy's error barcomes fromCategorical: a language model's softmaxoutputPoisson: check that the variance equals themeanExponential: memoryless, so waiting resetsnothingLog-normal: right-skewed, mean well abovemedianChoosing a distribution is a claim about how thedata were produced, not a question of which curvefits best. When counts show a variance far abovetheir mean, the Poisson story is wrong, and everyinterval built on it is far too narrow.

Which one, in one line each

  • Single yes/no: Bernoulli.
  • Count of yes out of n: binomial.
  • One of k options: categorical.
  • Count of rare events in a window: Poisson, and check the variance against the mean.
  • Gap between independent events: exponential, if nothing ages.
  • Positive, right-skewed, produced by multiplication: log-normal.
  • Sum of many small independent effects: normal, which the earlier lesson covers.

The honest caveat

Every one of these carries assumptions — independence, constant rate, identical distribution — that real data violates. Choosing a distribution is choosing a story, and the useful question is not "which fits best" but "which story is closest to how this actually happens, and how does it fail". Plot your data against the fitted distribution before relying on either. The mismatch in the tail is usually the interesting part, and it is invisible in a summary statistic.

The rule to keep

Name the generating process first, and the distribution follows. If you cannot state the process, you are curve-fitting, and the resulting probabilities in the tail — which is where the decisions are — will be wrong in ways the fit statistic will not show you.

The one thing to keep

Each standard distribution corresponds to a specific generating story, so choosing one is a claim about how your data was produced rather than a matter of which curve fits best.

Before you move on

A team models hourly error counts as Poisson to set an alerting threshold. Over 500 hours the mean is 12 and the variance is 140. What does that say, and what happens if they proceed?

Pick the one you would defend. Nobody sees your answer.

No ads. No data sale. No public scores on people. Ever.

© 2026 Addaly