{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"why-video-is-harder","title":"Why video is so much harder than a still","url":"https://addaly.com/learn/ai-actually-explained/why-video-is-harder","position":59,"of":92,"minutes":9,"answer":"Video asks hundreds of frames to agree about objects, light and physics the model never learned as rules, so the failures are objects that flicker into existence and liquids that behave impossibly.","keyIdea":"Video asks hundreds of frames to agree about objects, light and physics the model never learned as rules, so the failures are objects that flicker into existence and liquids that behave impossibly, at tens of cents a second."},"body":{"format":"markdown","text":"## Two hundred and forty pictures that must agree\n\nA still image is one frame. Ten seconds of video at twenty-four frames a second is two hundred and forty, and every one must agree with its neighbours about where each object is, how the light falls, what is behind what, and how things move. The chatbot module gave you the mechanism for why long sequences are hard; this is that mechanism with a camera attached.\n\nThe obvious approach (draw a frame, then draw the next one to match) fails quickly. Each frame introduces a small inconsistency, the next frame builds on it, and after a few seconds a person has drifted into a different person. The models that work do something closer to the image lesson: they treat the whole clip as one object, cut into patches across space *and* time, and denoise all of it together, so that frame ninety is drawn with frame one in view. It is enormously expensive. A ten-second clip is hundreds of times the work of a still, and the labs compress time the way the image models compressed space, working on a small grid and decoding to full frames at the end.\n\n## What you get in 2025\n\nThe leading commercial models produce eight to ten seconds of photographic footage from a sentence, with camera moves, in about a minute or two of data-centre time, and the best of them generate matching sound at the same time. Prices at launch were tens of cents per second of output. A prominent American lab's model was announced in February 2024 with the claim that it was a step toward simulating the physical world, and released to the public that December. Chinese models followed within months and, as with text, several were released as open weights.\n\n## The failures, and what they share\n\nWatch a few generated clips with the sound off and a list forms.\n\nA gymnast's limb detaches and rejoins. A glass shatters before the hand reaches it. Liquid pours upward, or fills a cup that stays empty. A person walks through a table. A wolf cub multiplies into three, then two. An object that leaves the frame comes back as a different object, or does not come back, or was never gone.\n\nEvery one of these is the same failure. The model learned what video *looks like* (how pixels tend to move between frames of the footage it saw) and not the world the footage was of. There is no object in the model to be permanent, no mass to fall, no glass to stay whole until struck. There is a statistical habit that things in videos usually keep looking the way they did, which holds for gentle motion and breaks the moment the physics gets interesting.\n\nIn early 2025 a research group built a benchmark of short real clips (a ball dropped, a candle blown, paint poured) and asked the leading models to continue them. It found that how realistic a model's output looked had almost no relationship to whether the physics in it was right. The models had become good at the surface, and the surface is what they were scored on.\n\nThat is the honest answer to the \"world simulator\" claim: not yet, and not by this route without something added. Whether more data closes the gap or whether the gap needs a different design is one of the open questions in the field.\n\n## What it is already used for\n\nWhere those failures do not matter, it is in daily use. Short advertising clips. Establishing shots. Animating a still product photograph. Backgrounds and B-roll for a video that a person then edits. Storyboards. Music videos where surreal is the brief. A great deal of what appears in feeds is now this, and the second module's advice about knowing who pays when it is wrong applies: a background is cheap to get wrong, a reconstruction of an event is not.\n\n## The free path\n\nOpen-weights video models from Chinese and Israeli labs since late 2024 run on a consumer graphics card; the smallest produces a few seconds at low resolution on eight gigabytes of memory, and all of them run in a free hosted notebook. The quality trails the closed models by months rather than years. For everything after generation (cutting, colour, sound, titles) DaVinci Resolve is free and used in professional post-production, with Shotcut and Kdenlive as lighter alternatives to the paid Premiere that job advertisements name; Blender is free for anything three-dimensional. And the same rule as for music applies: the model produces raw material, and the failures above are cut out in an editor, not prompted away."},"check":{"question":"A generated clip shows water being poured, and the cup stays empty while the stream continues. What does this failure tell you about how the model works?","options":[{"text":"The model ran out of steps before it could finish the pour.","correct":false,"tempting":"Treats the failure as incomplete rendering; the clip is fully rendered and the physics is wrong throughout."},{"text":"It learned how pixels tend to move between frames, not the world filmed, so nothing in it holds the water as a quantity.","correct":true},{"text":"The training data contained few videos of pouring, so the model had not seen enough examples of liquid filling a container.","correct":false,"tempting":"Blames scarce data for a failure that appears with abundant, everyday physics."},{"text":"The prompt did not specify that the cup should fill up.","correct":false,"tempting":"Assumes an unspecified outcome is undetermined; the failure is that the model has no representation of the outcome at all."}]},"prev":{"slug":"music-from-a-prompt","title":"Music from a prompt","url":"https://addaly.com/learn/ai-actually-explained/music-from-a-prompt"},"next":{"slug":"how-a-watermark-is-hidden","title":"How a watermark is hidden inside the output","url":"https://addaly.com/learn/ai-actually-explained/how-a-watermark-is-hidden"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Why video is so much harder than a still', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/why-video-is-harder (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Why video is so much harder than a still","courseTitle":"AI, Actually Explained","module":{"index":6,"slug":"beyond-text","title":"Beyond text: pictures, voices and video"},"answer":null,"keyIdea":"Video asks hundreds of frames to agree about objects, light and physics the model never learned as rules, so the failures are objects that flicker into existence and liquids that behave impossibly, at tens of cents a second.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-hundred-and-forty-pictures-that-must-agree","runs":[{"text":"Two hundred and forty pictures that must agree"}]},{"t":"p","runs":[{"text":"A still image is one frame. Ten seconds of video at twenty-four frames a second is two hundred and forty, and every one must agree with its neighbours about where each object is, how the light falls, what is behind what, and how things move. The chatbot module gave you the mechanism for why long sequences are hard; this is that mechanism with a camera attached."}]},{"t":"p","runs":[{"text":"The obvious approach (draw a frame, then draw the next one to match) fails quickly. Each frame introduces a small inconsistency, the next frame builds on it, and after a few seconds a person has drifted into a different person. The models that work do something closer to the image lesson: they treat the whole clip as one object, cut into patches across space "},{"text":"and","i":true},{"text":" time, and denoise all of it together, so that frame ninety is drawn with frame one in view. It is enormously expensive. A ten-second clip is hundreds of times the work of a still, and the labs compress time the way the image models compressed space, working on a small grid and decoding to full frames at the end."}]},{"t":"h2","id":"what-you-get-in-2025","runs":[{"text":"What you get in 2025"}]},{"t":"p","runs":[{"text":"The leading commercial models produce eight to ten seconds of photographic footage from a sentence, with camera moves, in about a minute or two of data-centre time, and the best of them generate matching sound at the same time. Prices at launch were tens of cents per second of output. A prominent American lab’s model was announced in February 2024 with the claim that it was a step toward simulating the physical world, and released to the public that December. Chinese models followed within months and, as with text, several were released as open weights."}]},{"t":"h2","id":"the-failures-and-what-they-share","runs":[{"text":"The failures, and what they share"}]},{"t":"p","runs":[{"text":"Watch a few generated clips with the sound off and a list forms."}]},{"t":"p","runs":[{"text":"A gymnast’s limb detaches and rejoins. A glass shatters before the hand reaches it. Liquid pours upward, or fills a cup that stays empty. A person walks through a table. A wolf cub multiplies into three, then two. An object that leaves the frame comes back as a different object, or does not come back, or was never gone."}]},{"t":"p","runs":[{"text":"Every one of these is the same failure. The model learned what video "},{"text":"looks like","i":true},{"text":" (how pixels tend to move between frames of the footage it saw) and not the world the footage was of. There is no object in the model to be permanent, no mass to fall, no glass to stay whole until struck. There is a statistical habit that things in videos usually keep looking the way they did, which holds for gentle motion and breaks the moment the physics gets interesting."}]},{"t":"p","runs":[{"text":"In early 2025 a research group built a benchmark of short real clips (a ball dropped, a candle blown, paint poured) and asked the leading models to continue them. It found that how realistic a model’s output looked had almost no relationship to whether the physics in it was right. The models had become good at the surface, and the surface is what they were scored on."}]},{"t":"p","runs":[{"text":"That is the honest answer to the “world simulator” claim: not yet, and not by this route without something added. Whether more data closes the gap or whether the gap needs a different design is one of the open questions in the field."}]},{"t":"h2","id":"what-it-is-already-used-for","runs":[{"text":"What it is already used for"}]},{"t":"p","runs":[{"text":"Where those failures do not matter, it is in daily use. Short advertising clips. Establishing shots. Animating a still product photograph. Backgrounds and B-roll for a video that a person then edits. Storyboards. Music videos where surreal is the brief. A great deal of what appears in feeds is now this, and the second module’s advice about knowing who pays when it is wrong applies: a background is cheap to get wrong, a reconstruction of an event is not."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"Open-weights video models from Chinese and Israeli labs since late 2024 run on a consumer graphics card; the smallest produces a few seconds at low resolution on eight gigabytes of memory, and all of them run in a free hosted notebook. The quality trails the closed models by months rather than years. For everything after generation (cutting, colour, sound, titles) DaVinci Resolve is free and used in professional post-production, with Shotcut and Kdenlive as lighter alternatives to the paid Premiere that job advertisements name; Blender is free for anything three-dimensional. And the same rule as for music applies: the model produces raw material, and the failures above are cut out in an editor, not prompted away."}]}],"check":{"question":[{"text":"A generated clip shows water being poured, and the cup stays empty while the stream continues. What does this failure tell you about how the model works?"}],"options":[{"text":[{"text":"The model ran out of steps before it could finish the pour."}],"correct":false,"tempting":[{"text":"Treats the failure as incomplete rendering; the clip is fully rendered and the physics is wrong throughout."}]},{"text":[{"text":"It learned how pixels tend to move between frames, not the world filmed, so nothing in it holds the water as a quantity."}],"correct":true,"tempting":null},{"text":[{"text":"The training data contained few videos of pouring, so the model had not seen enough examples of liquid filling a container."}],"correct":false,"tempting":[{"text":"Blames scarce data for a failure that appears with abundant, everyday physics."}]},{"text":[{"text":"The prompt did not specify that the cup should fill up."}],"correct":false,"tempting":[{"text":"Assumes an unspecified outcome is undetermined; the failure is that the model has no representation of the outcome at all."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"music-from-a-prompt","title":"Music from a prompt","minutes":8},"next":{"slug":"how-a-watermark-is-hidden","title":"How a watermark is hidden inside the output","minutes":9}}}