Nearly all of it is prediction

One mechanism, many products

A spam filter, a camera, a chatbot and a video feed look like four different technologies. Underneath, they are doing the same thing: taking what they can see and producing the most likely completion of it.

• The spam filter sees an email and predicts a label: spam or not.
• The camera sees a dim, noisy sensor reading and predicts what a sharper, brighter photograph of the same scene would have looked like.
• The chatbot sees the conversation so far and predicts the next chunk of text.
• The feed sees your history and predicts which video you will watch to the end.

Once you have the word prediction in place, the whole field stops looking like magic and starts looking like one idea applied in many directions. It also tells you where the failures will be, which is the more useful half.

How the prediction gets good

Training is a loop of three steps, repeated an enormous number of times.

1. The model makes a prediction on an example whose right answer is known.
2. The difference between the prediction and the right answer is measured. That difference has a name: the loss.
3. Every weight is nudged a tiny amount in the direction that would have made the loss smaller.

That is it. Do it a few trillion times across a few trillion examples and the loss goes down, which means the predictions get closer to the answers. The nudging procedure is called gradient descent, and the arithmetic behind it is first-year calculus, but you do not need the arithmetic to hold the shape: guess, compare, adjust, repeat.

You can watch this happen without any AI tooling at all. Open LibreOffice Calc or Google Sheets, put twenty points on a scatter chart, and add a trend line. The spreadsheet has just found the line that minimises the total squared distance to your points: the same three steps, with two numbers instead of eight billion. That trend line is a model. It predicts. It was fitted, not written. Everything else in this course is that, scaled up and stacked deep.

What prediction cannot do, by construction

Here is the limitation that follows directly from the mechanism, and it is the most important idea in this lesson.

A system trained to predict what happened will keep predicting what happened. It has no representation of what should happen. There is nothing in the loop where a value judgement could enter, because the only signal is the gap between the guess and the recorded answer.

This is not an abstract worry. A model trained to predict which released prisoners are arrested again is trained on arrest records, and arrest records reflect where police patrol as much as where crime occurs. The model learns the patrol pattern faithfully and reports it as risk. A model trained on a decade of a company’s hiring decisions learns that company’s decade of preferences, including the ones nobody would defend out loud. Amazon built such a screening tool in the mid-2010s, found it downgraded CVs containing the word “women’s”, and abandoned it.

Neither system malfunctioned. Both predicted the past accurately. That is the trap: the failure looks like success by every number the team was watching.

The second limitation: correlation is enough for the loss

The loop rewards anything that reduces the error, including features that are associated with the answer for reasons that have nothing to do with cause.

A well-known example from medical imaging: a model trained to spot pneumonia on chest X-rays learned to read the small metal token that portable scanners leave in the corner of the image. Portable scanners are wheeled to patients too ill to walk, so the token predicted illness beautifully, until the model met an X-ray from a different hospital and collapsed.

The model was not stupid. It found the cheapest reliable predictor available, which is precisely what the loop asks it to do.

What to do with this

Two questions, and they work on every system you will meet:

What exactly is it predicting? Not what it is for: what quantity is it actually trying to get right? A feed is not built to show you what is good; it is built to predict what you will watch. A hiring filter does not predict who will do the job well; it predicts who resembles people previously hired.

Where did the answers come from? Every training example has a recorded right answer, and somebody or some process produced it. That source is the ceiling on what the system can be. A model can be more consistent than the process that labelled its data. It cannot be wiser than it.