{"course":{"slug":"motion-and-audio","title":"Motion, Sound and Music","url":"https://addaly.com/learn/motion-and-audio","level":"intermediate","track":"create"},"lesson":{"slug":"generated-and-interpolated-frames","title":"Frames that were never photographed","url":"https://addaly.com/learn/motion-and-audio/generated-and-interpolated-frames","position":71,"of":72,"minutes":10,"answer":"Interpolation assumes every pixel has a correspondence in the neighbouring frame, so it tears at occlusion boundaries and fragments thin structures; and an upscaler does not reveal detail but fabricates it.","keyIdea":"Interpolation assumes every pixel has a correspondence in the neighbouring frame, so it tears at occlusion boundaries and fragments thin structures; and an upscaler does not reveal detail but fabricates it, which makes upscaled output unusable as a record of anything."},"body":{"format":"markdown","text":"## Frames that were never photographed\n\nThree related operations invent picture information that did not exist: **interpolation** makes new frames between real ones, **upscaling** makes new pixels within a frame, and **video generation** makes everything. All three are estimation, and all three fail in ways worth being able to name before you rely on them.\n\n## Interpolation, and where it breaks\n\nClassical frame interpolation estimates **optical flow** (a vector per pixel describing where it moved between two frames) and warps the two frames toward an intermediate time. Learned methods such as RIFE and FILM replace the hand-built flow estimator with a network, and generally produce better results on difficult motion.\n\nBoth fail in the same four places, and all four follow from the same cause: flow assumes every pixel in one frame has a corresponding pixel in the other.\n\n- **Occlusion boundaries.** Where something appears from behind something else, the newly revealed pixels have no correspondence in the earlier frame. The estimator has to invent them, and the result is a smearing or tearing around the edges of moving objects. This is the most common visible artefact.\n- **Thin structures.** A railing, a wire, a spoke, a strand of hair. A thin object can move further than its own width between frames, so there is no overlap to match, and it breaks into fragments or disappears.\n- **Repetitive texture.** A picket fence, brickwork, a striped shirt. Many candidate matches score equally well, and the flow field picks inconsistently across the region, producing a rippling wobble.\n- **Large motion and blur.** Fast pans and heavily blurred frames give the estimator nothing sharp to correspond.\n\nThe useful rule: interpolation works well on smooth, moderate motion of large solid objects, and badly on everything else. For genuine slow motion, shoot at the higher rate; the frame-rate lesson makes the same point and it is the same reason.\n\n## Upscaling invents plausible detail\n\nA super-resolution model has learned what high-resolution images look like, and given a low-resolution input it produces a high-resolution image consistent with it. Consistent is not the same as correct.\n\nThe consequence needs stating plainly: **an upscaler does not reveal detail, it fabricates detail**. If a number plate is six pixels across, the information identifying its characters is not in the file. An upscaler will produce sharp, legible characters, and they will be a plausible guess. Any use that treats upscaled output as evidence (forensic, journalistic, identifying a person) is treating an invention as a measurement.\n\nThe characteristic artefacts:\n\n- **Hallucinated text.** Small illegible lettering becomes sharp nonsense, or sharp wrong words.\n- **Faces.** Models trained heavily on faces reconstruct a face, which can be a different person's face. This has real consequences and has been demonstrated repeatedly.\n- **Over-sharpened texture.** Skin becomes plastic, foliage becomes a repeating pattern, film grain becomes a crawling texture.\n- **Temporal instability on video.** A per-frame model reconstructs slightly differently each frame, so the invented detail shimmers. Video-specific models reduce this and do not eliminate it.\n\nWhere upscaling is legitimately useful: restoring archive material for viewing, enlarging for a delivery format, cleaning a compressed source. Where it is not: anything where the detail is the point.\n\n## Generated video\n\nText-to-video and image-to-video models produce clips of a few seconds. The state of the art moves quickly and the structural limitations have been persistent:\n\n- **Temporal consistency.** Objects drift, change colour, and lose or gain parts across a clip. There is no persistent representation of a scene.\n- **Physics.** Contact, weight and momentum are approximated from appearance rather than modelled. Things pass through each other and liquids behave oddly.\n- **Hands, text and counting.** All three remain unreliable, for related reasons: they require a structural constraint the model has learned only statistically.\n- **Duration.** Coherence degrades with length, which is why outputs are short and why longer pieces are assembled from shots.\n\nTreated as a source of short, controllable elements inside an edit (a texture, a background plate, an abstract transition) this is useful. Treated as a replacement for shooting a sequence, it is not there.\n\n## Judging any of the three\n\nPlay it at full speed, and play it twice. All three failure classes are temporal: a frame-by-frame inspection shows nothing, and the wobble, the shimmer and the drift only exist in motion.\n\nThen look at the specific places named above (occlusion edges, thin objects, text, hands, repeated texture) rather than at the picture in general. The artefacts are predictable, so you can check for them directly.\n\n## The free path\n\n- **RIFE** and **FILM**: open-source interpolation, run locally, both available as command-line tools and as plugins for free editors.\n- **ffmpeg**'s `minterpolate` filter: classical motion-compensated interpolation, free and built in, lower quality than the learned methods but present everywhere.\n- **Real-ESRGAN**, **BasicVSR++**, **waifu2x**: open-source upscalers, including video-specific ones.\n- **DaVinci Resolve**'s free edition has optical-flow retiming and a super-scale function.\n- Open-weight video generation models exist and change too fast for a list here to be useful; check their licences, which frequently restrict commercial use.\n\n## The limitation\n\nAll three operations share one property: they produce confident output with no signal about how much of it is invented. There is no per-pixel uncertainty map in the file, no warning that this region was extrapolated, no distinction in the output between a pixel derived from the source and one produced from the prior.\n\nThat is manageable for entertainment and dangerous everywhere else. The discipline that follows is to keep the original, to say in any handover that the material has been interpolated or upscaled, and never to let a generated frame be used as a record of something that happened."},"check":{"question":"Why does frame interpolation produce its most visible artefacts around the edges of moving objects rather than in their interiors?","options":[{"text":"Edges contain the highest spatial frequencies, and the interpolator's warping operation attenuates high frequencies more than low ones.","correct":false,"tempting":"Warping does soften detail, but softening is not the tearing and smearing seen at moving edges."},{"text":"Where something is revealed from behind something else, the newly visible pixels have no counterpart in the earlier frame, so there is nothing to warp and the model has to invent them.","correct":true},{"text":"Optical flow is estimated on a coarse grid and refined, and the refinement stage cannot resolve motion boundaries finer than the coarsest grid cell.","correct":false,"tempting":"Coarse-to-fine estimation does blur motion boundaries, but the fundamental problem is missing correspondence rather than grid resolution."},{"text":"Objects near an edge are usually at a different depth from the background, and any depth discontinuity means the two regions move by different amounts, which violates the smoothness assumption that flow estimation relies on and causes the vectors in that neighbourhood to be averaged into a single incorrect value.","correct":false,"tempting":"Describes the smoothness-violation half of the problem accurately, but the irreducible issue is that disoccluded pixels have no source to come from at all."}]},"prev":{"slug":"what-a-licence-covers","title":"What a music licence actually covers","url":"https://addaly.com/learn/motion-and-audio/what-a-licence-covers"},"next":{"slug":"provenance-and-disclosure","title":"Provenance, watermarks and what to tell people","url":"https://addaly.com/learn/motion-and-audio/provenance-and-disclosure"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Frames that were never photographed', in Motion, Sound and Music. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/motion-and-audio/generated-and-interpolated-frames (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Frames that were never photographed","courseTitle":"Motion, Sound and Music","module":{"index":8,"slug":"machines-that-make-sound-and-motion","title":"Machines that make sound and motion"},"answer":null,"keyIdea":"Interpolation assumes every pixel has a correspondence in the neighbouring frame, so it tears at occlusion boundaries and fragments thin structures; and an upscaler does not reveal detail but fabricates it, which makes upscaled output unusable as a record of anything.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"frames-that-were-never-photographed","runs":[{"text":"Frames that were never photographed"}]},{"t":"p","runs":[{"text":"Three related operations invent picture information that did not exist: "},{"text":"interpolation","b":true},{"text":" makes new frames between real ones, "},{"text":"upscaling","b":true},{"text":" makes new pixels within a frame, and "},{"text":"video generation","b":true},{"text":" makes everything. All three are estimation, and all three fail in ways worth being able to name before you rely on them."}]},{"t":"h2","id":"interpolation-and-where-it-breaks","runs":[{"text":"Interpolation, and where it breaks"}]},{"t":"p","runs":[{"text":"Classical frame interpolation estimates "},{"text":"optical flow","b":true},{"text":" (a vector per pixel describing where it moved between two frames) and warps the two frames toward an intermediate time. Learned methods such as RIFE and FILM replace the hand-built flow estimator with a network, and generally produce better results on difficult motion."}]},{"t":"p","runs":[{"text":"Both fail in the same four places, and all four follow from the same cause: flow assumes every pixel in one frame has a corresponding pixel in the other."}]},{"t":"ul","items":[[{"text":"Occlusion boundaries.","b":true},{"text":" Where something appears from behind something else, the newly revealed pixels have no correspondence in the earlier frame. The estimator has to invent them, and the result is a smearing or tearing around the edges of moving objects. This is the most common visible artefact."}],[{"text":"Thin structures.","b":true},{"text":" A railing, a wire, a spoke, a strand of hair. A thin object can move further than its own width between frames, so there is no overlap to match, and it breaks into fragments or disappears."}],[{"text":"Repetitive texture.","b":true},{"text":" A picket fence, brickwork, a striped shirt. Many candidate matches score equally well, and the flow field picks inconsistently across the region, producing a rippling wobble."}],[{"text":"Large motion and blur.","b":true},{"text":" Fast pans and heavily blurred frames give the estimator nothing sharp to correspond."}]]},{"t":"p","runs":[{"text":"The useful rule: interpolation works well on smooth, moderate motion of large solid objects, and badly on everything else. For genuine slow motion, shoot at the higher rate; the frame-rate lesson makes the same point and it is the same reason."}]},{"t":"h2","id":"upscaling-invents-plausible-detail","runs":[{"text":"Upscaling invents plausible detail"}]},{"t":"p","runs":[{"text":"A super-resolution model has learned what high-resolution images look like, and given a low-resolution input it produces a high-resolution image consistent with it. Consistent is not the same as correct."}]},{"t":"p","runs":[{"text":"The consequence needs stating plainly: "},{"text":"an upscaler does not reveal detail, it fabricates detail","b":true},{"text":". If a number plate is six pixels across, the information identifying its characters is not in the file. An upscaler will produce sharp, legible characters, and they will be a plausible guess. Any use that treats upscaled output as evidence (forensic, journalistic, identifying a person) is treating an invention as a measurement."}]},{"t":"p","runs":[{"text":"The characteristic artefacts:"}]},{"t":"ul","items":[[{"text":"Hallucinated text.","b":true},{"text":" Small illegible lettering becomes sharp nonsense, or sharp wrong words."}],[{"text":"Faces.","b":true},{"text":" Models trained heavily on faces reconstruct a face, which can be a different person’s face. This has real consequences and has been demonstrated repeatedly."}],[{"text":"Over-sharpened texture.","b":true},{"text":" Skin becomes plastic, foliage becomes a repeating pattern, film grain becomes a crawling texture."}],[{"text":"Temporal instability on video.","b":true},{"text":" A per-frame model reconstructs slightly differently each frame, so the invented detail shimmers. Video-specific models reduce this and do not eliminate it."}]]},{"t":"p","runs":[{"text":"Where upscaling is legitimately useful: restoring archive material for viewing, enlarging for a delivery format, cleaning a compressed source. Where it is not: anything where the detail is the point."}]},{"t":"h2","id":"generated-video","runs":[{"text":"Generated video"}]},{"t":"p","runs":[{"text":"Text-to-video and image-to-video models produce clips of a few seconds. The state of the art moves quickly and the structural limitations have been persistent:"}]},{"t":"ul","items":[[{"text":"Temporal consistency.","b":true},{"text":" Objects drift, change colour, and lose or gain parts across a clip. There is no persistent representation of a scene."}],[{"text":"Physics.","b":true},{"text":" Contact, weight and momentum are approximated from appearance rather than modelled. Things pass through each other and liquids behave oddly."}],[{"text":"Hands, text and counting.","b":true},{"text":" All three remain unreliable, for related reasons: they require a structural constraint the model has learned only statistically."}],[{"text":"Duration.","b":true},{"text":" Coherence degrades with length, which is why outputs are short and why longer pieces are assembled from shots."}]]},{"t":"p","runs":[{"text":"Treated as a source of short, controllable elements inside an edit (a texture, a background plate, an abstract transition) this is useful. Treated as a replacement for shooting a sequence, it is not there."}]},{"t":"h2","id":"judging-any-of-the-three","runs":[{"text":"Judging any of the three"}]},{"t":"p","runs":[{"text":"Play it at full speed, and play it twice. All three failure classes are temporal: a frame-by-frame inspection shows nothing, and the wobble, the shimmer and the drift only exist in motion."}]},{"t":"p","runs":[{"text":"Then look at the specific places named above (occlusion edges, thin objects, text, hands, repeated texture) rather than at the picture in general. The artefacts are predictable, so you can check for them directly."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"ul","items":[[{"text":"RIFE","b":true},{"text":" and "},{"text":"FILM","b":true},{"text":": open-source interpolation, run locally, both available as command-line tools and as plugins for free editors."}],[{"text":"ffmpeg","b":true},{"text":"’s "},{"text":"minterpolate","code":true},{"text":" filter: classical motion-compensated interpolation, free and built in, lower quality than the learned methods but present everywhere."}],[{"text":"Real-ESRGAN","b":true},{"text":", "},{"text":"BasicVSR++","b":true},{"text":", "},{"text":"waifu2x","b":true},{"text":": open-source upscalers, including video-specific ones."}],[{"text":"DaVinci Resolve","b":true},{"text":"’s free edition has optical-flow retiming and a super-scale function."}],[{"text":"Open-weight video generation models exist and change too fast for a list here to be useful; check their licences, which frequently restrict commercial use."}]]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"All three operations share one property: they produce confident output with no signal about how much of it is invented. There is no per-pixel uncertainty map in the file, no warning that this region was extrapolated, no distinction in the output between a pixel derived from the source and one produced from the prior."}]},{"t":"p","runs":[{"text":"That is manageable for entertainment and dangerous everywhere else. The discipline that follows is to keep the original, to say in any handover that the material has been interpolated or upscaled, and never to let a generated frame be used as a record of something that happened."}]}],"check":{"question":[{"text":"Why does frame interpolation produce its most visible artefacts around the edges of moving objects rather than in their interiors?"}],"options":[{"text":[{"text":"Edges contain the highest spatial frequencies, and the interpolator’s warping operation attenuates high frequencies more than low ones."}],"correct":false,"tempting":[{"text":"Warping does soften detail, but softening is not the tearing and smearing seen at moving edges."}]},{"text":[{"text":"Where something is revealed from behind something else, the newly visible pixels have no counterpart in the earlier frame, so there is nothing to warp and the model has to invent them."}],"correct":true,"tempting":null},{"text":[{"text":"Optical flow is estimated on a coarse grid and refined, and the refinement stage cannot resolve motion boundaries finer than the coarsest grid cell."}],"correct":false,"tempting":[{"text":"Coarse-to-fine estimation does blur motion boundaries, but the fundamental problem is missing correspondence rather than grid resolution."}]},{"text":[{"text":"Objects near an edge are usually at a different depth from the background, and any depth discontinuity means the two regions move by different amounts, which violates the smoothness assumption that flow estimation relies on and causes the vectors in that neighbourhood to be averaged into a single incorrect value."}],"correct":false,"tempting":[{"text":"Describes the smoothness-violation half of the problem accurately, but the irreducible issue is that disoccluded pixels have no source to come from at all."}]}],"order":[2,3,0,1],"why":null},"prev":{"slug":"what-a-licence-covers","title":"What a music licence actually covers","minutes":11},"next":{"slug":"provenance-and-disclosure","title":"Provenance, watermarks and what to tell people","minutes":10}}}