The whole idea, in one paragraph
A model starts as an enormous list of numbers set at random. Call them dials. You show the model one example and it produces an answer, which at first is nonsense. You compare that answer to the correct one from your examples, and you nudge every dial a tiny amount in whatever direction would have made the answer slightly less wrong. Then you do it again with the next example. And again, a few billion times.
That is training. There is no other secret step. The intelligence, such as it is, is what accumulates in the dials.
A small version you can hold in your head
Suppose you want a system that estimates a flat's rent from its size, floor and neighbourhood. You have twenty thousand real listings with real rents.
Show it the first flat. It guesses 90,000 pesos. The real rent was 22,000. That is badly wrong, so nudge the dials.
Show it the next. It guesses 30,000, the real answer is 25,000. Closer. Nudge less.
After twenty thousand flats, several times over, the guesses are usually within a reasonable range. Nothing about property or cities was explained to it. It found the pattern in the examples because being closer to the examples is the only thing it was ever pushed toward.
Large models work the same way. They are much bigger, the examples are text and images rather than rents, and the nudging costs millions of dollars in electricity. The idea does not change.
Then the examples are everything
This is the consequence that matters most and gets skipped most.
A model has no source of behaviour other than what it was shown. If the examples are lopsided, the model is lopsided, and it will be lopsided smoothly and confidently, with no warning label.
A speech system trained mostly on American and British voices will do noticeably worse on a Malaysian or Nigerian accent, and will not tell you it is struggling. A skin-condition model trained mostly on light skin performs worse on dark skin. A model trained on a company's past hiring decisions learns who that company used to hire, which is not the same as who was good at the job.
None of these need a villain. Nobody wrote a rule. The system reproduced its examples faithfully, which is exactly what it was built to do.
Two things people usually get wrong
Training is not memorising a database. The model does not keep the examples. It keeps the dials. This is why it can answer questions nobody asked it before, and also why it cannot reliably quote a source. There are exceptions: text that appeared thousands of times, like a famous poem or a common licence, often does come back word for word.
Training happens before you arrive. It is a huge, one-off, expensive process. Using the model afterwards is cheap and does not change a single dial. When you correct a chatbot, you have changed nothing about the model. More on that later.
Where the examples come from
For large text models, the answer is mostly the public internet, plus books, code and licensed collections, plus text written or rated by paid human workers. This is genuinely contested. Writers, artists and publishers are in court over whether training on their work without permission is allowed, and different countries are landing in different places. There is no settled answer to give you, and anyone who says there is has picked a side.
Before you move on