{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"the-chip-built-for-games","title":"The chip that was built for games","url":"https://addaly.com/learn/ai-actually-explained/the-chip-built-for-games","position":46,"of":92,"minutes":9,"answer":"Training a network is almost entirely multiplying large grids of numbers, and a graphics chip is thousands of small cores built to do exactly that at once.","keyIdea":"Training a network is almost entirely multiplying large grids of numbers, and a graphics chip is thousands of small cores built to do exactly that at once, which is why one games company came to sit under the whole industry."},"body":{"format":"markdown","text":"## Two ways to be fast\n\nA processor in a laptop has a handful of cores, each of them fast, each able to do something different from the others, and each spending most of its transistor budget on deciding what to do next. That is the right design for running a word processor, where the work is a long chain of different small decisions.\n\nA graphics chip is the opposite. It has thousands of small cores that all do the same simple arithmetic at the same time on different data. It was built that way because a screen has millions of pixels and each one needs the same shading calculation. Nobody designed it for intelligence. It was designed so a game could draw a wall at sixty frames a second.\n\n## Why that is the shape of training\n\nStrip a neural network down and almost all of the work is one operation: multiply a grid of numbers by another grid of numbers and add up the results. The grids are the weights you met in the first module and the activations flowing through them. A training run does this operation trillions of times, and each multiplication inside it is independent of the others.\n\nIndependent, identical, arithmetic: that is precisely what a graphics chip does. The two gaming cards in the 2012 result were not a clever hack. They were the first time anyone had put the right-shaped machine under the right-shaped problem.\n\nThe decision that made it possible was commercial and dates from 2007. That year Nvidia released a way for ordinary programmers to write general calculations for its chips rather than only graphics. Scientists simulating weather and proteins used it first. The AI researchers came five years later, and found a mature tool waiting. That one decision is a large part of why a single company now supplies, by most estimates, four fifths or more of the chips used to train and serve models.\n\n## The numbers\n\nThe 2010 gaming card in that first result could do about one and a half trillion arithmetic operations a second. The A100, released in 2020 for data centres, does around three hundred trillion on the reduced-precision numbers training uses. Its successor, the H100 of 2022, roughly triples that again. A single H100 sells for tens of thousands of dollars and draws around seven hundred watts, about the same as a small electric heater.\n\n> Figure: Arithmetic per second, from a gaming card to a data-centre chip. Bars (trillion reduced-precision operations per second): GTX 580, 2010 gaming card 1.5; A100, 2020 data centre 312; H100, 2022 data centre 990. Training is almost entirely multiplying grids of numbers, and each multiplication is independent of the others: the shape a graphics chip was built for, to shade millions of pixels at once. The 2012 result that turned the field used two of the first bar. A single H100 costs tens of thousands of dollars and draws about 700 watts.\n\nA frontier model is not trained on one. Reports put the cluster used for GPT-4 at around twenty-five thousand A100s. In 2024 one lab connected a hundred thousand H100s in a single building. At that scale the constraint is no longer arithmetic; it is moving data between the chips and keeping the building cool and powered, which is why the electricity questions in the last module of this course are not a side issue.\n\n## The real bottleneck is memory\n\nA detail that explains a great deal of pricing. The arithmetic on a modern chip is so fast that it spends much of its time waiting for numbers to arrive from memory. The memory is stacked next to the chip and is the expensive, scarce part. When you hear that a model \"fits\" on a card, it means its weights fit in that memory; a model of seventy billion parameters at sixteen bits per number needs about a hundred and forty gigabytes, which is more than one H100 holds. That is why large models are spread across several cards, why serving them costs what it does, and why the module on running a model on your own machine will spend its time on shrinking the numbers.\n\n## Who else makes them\n\nGoogle has built its own chips since 2016 and trains its models on them. Amazon and Microsoft have followed. Every recent phone carries a small version for the camera and the keyboard. And since 2022 the United States has restricted which of these chips can be sold to China, which is why a Chinese lab's model in early 2025 was trained on a deliberately cut-down variant and why the question of who owns the chips is now part of foreign policy.\n\n## The free path\n\nYou do not need one to learn. Google Colab gives a free notebook with a shared data-centre GPU for a few hours at a time; Kaggle offers a weekly allowance of the same. Both are enough to fine-tune a small model or train a classifier. A model small enough to be useful runs on the processor you already have, slowly, as a later module shows.\n\n## What a graphics chip is bad at\n\nIt is bad at the kind of work a laptop processor is good at: long chains of decisions where each depends on the last. That is one mechanism behind a limit you have already met. Generating text is sequential (each token depends on the previous one) so a chip built for doing a million things at once is fed one token at a time and spends most of its capacity idle. Training is parallel and fast; answering you is sequential and comparatively slow. When a model \"thinks step by step\", every step is another pass through that bottleneck, and you pay for it in seconds."},"check":{"question":"Why is a chip designed to draw video-game graphics well suited to training a neural network?","options":[{"text":"Because its clock runs faster than a laptop processor's.","correct":false,"tempting":"Confuses speed per core with the number of cores; a graphics chip's cores are individually slower."},{"text":"Because it runs thousands of identical multiplications at once, which is nearly all training is.","correct":true},{"text":"Because it carries more memory than a laptop processor.","correct":false,"tempting":"Picks the wrong resource; memory is the scarce part of these chips, not the abundant one."},{"text":"Because Nvidia designed the chip from the start with neural-network training in mind and added graphics later.","correct":false,"tempting":"Reads the history backwards; the chips were built for games, and general programming was added in 2007."}]},"prev":{"slug":"the-2012-moment","title":"The photograph competition that turned the field","url":"https://addaly.com/learn/ai-actually-explained/the-2012-moment"},"next":{"slug":"one-architecture-ate-everything","title":"One architecture ate everything","url":"https://addaly.com/learn/ai-actually-explained/one-architecture-ate-everything"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The chip that was built for games', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/the-chip-built-for-games (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The chip that was built for games","courseTitle":"AI, Actually Explained","module":{"index":5,"slug":"how-it-got-here","title":"How it got here, and what changed in 2022"},"answer":null,"keyIdea":"Training a network is almost entirely multiplying large grids of numbers, and a graphics chip is thousands of small cores built to do exactly that at once, which is why one games company came to sit under the whole industry.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"trillion reduced-precision operations per second","title":"Arithmetic per second, from a gaming card to a data-centre chip","series":[{"label":"GTX 580, 2010 gaming card","value":1.5},{"label":"A100, 2020 data centre","value":312},{"label":"H100, 2022 data centre","value":990,"highlight":true}],"caption":"Training is almost entirely multiplying grids of numbers, and each multiplication is independent of the others: the shape a graphics chip was built for, to shade millions of pixels at once. The 2012 result that turned the field used two of the first bar. A single H100 costs tens of thousands of dollars and draws about 700 watts."},"alt":"Figure: Arithmetic per second, from a gaming card to a data-centre chip. Bars (trillion reduced-precision operations per second): GTX 580, 2010 gaming card 1.5; A100, 2020 data centre 312; H100, 2022 data centre 990. Training is almost entirely multiplying grids of numbers, and each multiplication is independent of the others: the shape a graphics chip was built for, to shade millions of pixels at once. The 2012 result that turned the field used two of the first bar. A single H100 costs tens of thousands of dollars and draws about 700 watts."}},"blocks":[{"t":"h2","id":"two-ways-to-be-fast","runs":[{"text":"Two ways to be fast"}]},{"t":"p","runs":[{"text":"A processor in a laptop has a handful of cores, each of them fast, each able to do something different from the others, and each spending most of its transistor budget on deciding what to do next. That is the right design for running a word processor, where the work is a long chain of different small decisions."}]},{"t":"p","runs":[{"text":"A graphics chip is the opposite. It has thousands of small cores that all do the same simple arithmetic at the same time on different data. It was built that way because a screen has millions of pixels and each one needs the same shading calculation. Nobody designed it for intelligence. It was designed so a game could draw a wall at sixty frames a second."}]},{"t":"h2","id":"why-that-is-the-shape-of-training","runs":[{"text":"Why that is the shape of training"}]},{"t":"p","runs":[{"text":"Strip a neural network down and almost all of the work is one operation: multiply a grid of numbers by another grid of numbers and add up the results. The grids are the weights you met in the first module and the activations flowing through them. A training run does this operation trillions of times, and each multiplication inside it is independent of the others."}]},{"t":"p","runs":[{"text":"Independent, identical, arithmetic: that is precisely what a graphics chip does. The two gaming cards in the 2012 result were not a clever hack. They were the first time anyone had put the right-shaped machine under the right-shaped problem."}]},{"t":"p","runs":[{"text":"The decision that made it possible was commercial and dates from 2007. That year Nvidia released a way for ordinary programmers to write general calculations for its chips rather than only graphics. Scientists simulating weather and proteins used it first. The AI researchers came five years later, and found a mature tool waiting. That one decision is a large part of why a single company now supplies, by most estimates, four fifths or more of the chips used to train and serve models."}]},{"t":"h2","id":"the-numbers","runs":[{"text":"The numbers"}]},{"t":"p","runs":[{"text":"The 2010 gaming card in that first result could do about one and a half trillion arithmetic operations a second. The A100, released in 2020 for data centres, does around three hundred trillion on the reduced-precision numbers training uses. Its successor, the H100 of 2022, roughly triples that again. A single H100 sells for tens of thousands of dollars and draws around seven hundred watts, about the same as a small electric heater."}]},{"t":"p","runs":[{"text":"A frontier model is not trained on one. Reports put the cluster used for GPT-4 at around twenty-five thousand A100s. In 2024 one lab connected a hundred thousand H100s in a single building. At that scale the constraint is no longer arithmetic; it is moving data between the chips and keeping the building cool and powered, which is why the electricity questions in the last module of this course are not a side issue."}]},{"t":"h2","id":"the-real-bottleneck-is-memory","runs":[{"text":"The real bottleneck is memory"}]},{"t":"p","runs":[{"text":"A detail that explains a great deal of pricing. The arithmetic on a modern chip is so fast that it spends much of its time waiting for numbers to arrive from memory. The memory is stacked next to the chip and is the expensive, scarce part. When you hear that a model “fits” on a card, it means its weights fit in that memory; a model of seventy billion parameters at sixteen bits per number needs about a hundred and forty gigabytes, which is more than one H100 holds. That is why large models are spread across several cards, why serving them costs what it does, and why the module on running a model on your own machine will spend its time on shrinking the numbers."}]},{"t":"h2","id":"who-else-makes-them","runs":[{"text":"Who else makes them"}]},{"t":"p","runs":[{"text":"Google has built its own chips since 2016 and trains its models on them. Amazon and Microsoft have followed. Every recent phone carries a small version for the camera and the keyboard. And since 2022 the United States has restricted which of these chips can be sold to China, which is why a Chinese lab’s model in early 2025 was trained on a deliberately cut-down variant and why the question of who owns the chips is now part of foreign policy."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"You do not need one to learn. Google Colab gives a free notebook with a shared data-centre GPU for a few hours at a time; Kaggle offers a weekly allowance of the same. Both are enough to fine-tune a small model or train a classifier. A model small enough to be useful runs on the processor you already have, slowly, as a later module shows."}]},{"t":"h2","id":"what-a-graphics-chip-is-bad-at","runs":[{"text":"What a graphics chip is bad at"}]},{"t":"p","runs":[{"text":"It is bad at the kind of work a laptop processor is good at: long chains of decisions where each depends on the last. That is one mechanism behind a limit you have already met. Generating text is sequential (each token depends on the previous one) so a chip built for doing a million things at once is fed one token at a time and spends most of its capacity idle. Training is parallel and fast; answering you is sequential and comparatively slow. When a model “thinks step by step”, every step is another pass through that bottleneck, and you pay for it in seconds."}]}],"check":{"question":[{"text":"Why is a chip designed to draw video-game graphics well suited to training a neural network?"}],"options":[{"text":[{"text":"Because its clock runs faster than a laptop processor’s."}],"correct":false,"tempting":[{"text":"Confuses speed per core with the number of cores; a graphics chip’s cores are individually slower."}]},{"text":[{"text":"Because it runs thousands of identical multiplications at once, which is nearly all training is."}],"correct":true,"tempting":null},{"text":[{"text":"Because it carries more memory than a laptop processor."}],"correct":false,"tempting":[{"text":"Picks the wrong resource; memory is the scarce part of these chips, not the abundant one."}]},{"text":[{"text":"Because Nvidia designed the chip from the start with neural-network training in mind and added graphics later."}],"correct":false,"tempting":[{"text":"Reads the history backwards; the chips were built for games, and general programming was added in 2007."}]}],"order":[3,2,1,0],"why":null},"prev":{"slug":"the-2012-moment","title":"The photograph competition that turned the field","minutes":9},"next":{"slug":"one-architecture-ate-everything","title":"One architecture ate everything","minutes":9}}}