{"course":{"slug":"ai-video","title":"Video with AI","url":"https://addaly.com/learn/ai-video","level":"intermediate","track":"create"},"lesson":{"slug":"sound-design-fixes-bad-physics","title":"The viewer believes the sound","url":"https://addaly.com/learn/ai-video/sound-design-fixes-bad-physics","position":47,"of":66,"minutes":8,"answer":"When sound and picture disagree, perception leans on the sound.","keyIdea":"When sound and picture disagree, perception leans on the sound, which is why a well-synced impact makes a physically wrong collision read as correct and why sound design is a repair tool as well as a finishing one."},"body":{"format":"markdown","text":"## Audio wins arguments with vision\n\nThis is not a metaphor. It is a measurable perceptual effect.\n\nThe McGurk effect is the famous demonstration: a video of a mouth saying one syllable dubbed with a different syllable is heard as a third thing that neither source contains. Vision and hearing are being combined, and the combination is involuntary: knowing about the effect does not switch it off.\n\nRelated findings run through the whole perception literature. Sound changes the perceived timing of visual events. Sound changes the perceived force of an impact. A single flash accompanied by two beeps is often perceived as two flashes.\n\nFor generative video this has a direct, exploitable consequence: **a sound can repair a picture**.\n\n## Where this pays\n\nRecall the list of things the model cannot do. Several of them are substantially fixable with audio.\n\n**Impacts and collisions.** A glass hitting a floor and breaking into the wrong number of pieces is one of the model's weaknesses. Put a real, sharp, correctly-timed breaking-glass sound on the exact frame of contact and the shot reads. The eye had a moment of ambiguity and the ear resolved it.\n\n**Footsteps.** Generated walking often has slightly wrong weight: the foot lands and the body does not respond correctly. Footsteps recorded on the right surface, placed on the contact frames, supply the weight the picture lacks.\n\n**Contact events generally.** A hand touching an object, a door closing, a cup set down. These are exactly the moments the model handles worst, and they are also the moments that carry the clearest sounds.\n\n**Off-screen causes.** The strongest version. A sound implies an event the picture never had to show. A car crash heard over a reaction shot is a car crash, and you generated nothing.\n\n## The technique, precisely\n\nThree things and they are all about precision.\n\n1. **Find the exact frame of contact.** Step through the clip frame by frame. The contact frame is usually one or two earlier than you think, because you are reacting to the frame where the change is obvious rather than where it began.\n2. **Place the transient on that frame.** The transient is the sharp attack at the start of the sound, which is not necessarily the start of the file. Zoom into the waveform and align the spike, not the edge.\n3. **Match the sound's character to the picture.** A light object needs a light sound. Nothing exposes a fake more than a heavy impact on a small event, and this is worth watching for, because stock libraries are full of exaggerated sounds designed for trailers.\n\nTwo frames late reads as wrong. On the frame, it disappears into the picture and does its work.\n\n## What sound cannot fix\n\nHonesty about the limit. Sound resolves ambiguity; it does not overwrite something clearly seen.\n\n- A hand with six fingers is not fixed by any sound.\n- A face that changes identity mid-shot is not fixed by any sound.\n- Text that melts is not fixed by any sound.\n- Slow motion defeats the whole technique. The audience has been given time to look, the ambiguity is resolved visually, and no audio cue competes with that. This is the mechanism behind the earlier advice never to put fake physics in slow motion.\n\nThe rule: sound repairs fast, ambiguous, brief events. It does nothing for sustained, clearly visible errors.\n\n> Figure: What a sound on the right frame repairs. Repaired by a well-placed sound: A glass breaking into the wrong number of pieces, at speed; Footsteps whose weight does not match the body above them; A hand meeting an object, a door closing, a cup set down; An event kept off screen and carried by a reaction shot. Repaired by nothing you can add: A hand with six fingers; A face that changes identity mid-shot; Text melting across a sign; The same impact played at forty per cent speed. Sound resolves ambiguity; it does not overwrite something clearly seen. Slow motion removes the ambiguity by giving the viewer time to check, which is the mechanism behind the rule never to put generated physics in slow motion.\n\n## Designing the cut around this\n\nOnce you know sound carries impacts, you can build shots that rely on it:\n\n- Cut away *at* the contact and let the sound continue over the next shot. The audience assembles an event they never saw. This is the oldest trick in the medium and it works perfectly here.\n- Put difficult physics off-screen entirely and cover it with sound plus a reaction.\n- Let a sound start before its picture (an L-cut) so the next shot arrives already explained.\n\nThese are standard editing moves. They are worth restating here because in generative work they are not merely stylish, they are how you get shots you could not otherwise generate.\n\n## The free path\n\nEverything: Audacity to trim and shape, Freesound or your own phone for material, and any editor with frame-accurate placement. Resolve's free tier shows waveforms at frame level. Kdenlive and Shotcut do too.\n\nThe skill is precision, and precision costs attention rather than money.\n\n---\n\nToday: take a generated clip with a contact event that looks slightly wrong. Find the contact frame, place one real sound on it, and watch it again."},"check":{"question":"A generated shot of a mug being set down on a table looks slightly wrong: the contact reads as weightless. Adding a recorded mug-on-wood sound on the contact frame fixes the impression. Why does this work?","options":[{"text":"The sound draws attention away from the contact moment, so the viewer's eye is elsewhere when the error occurs.","correct":false,"tempting":"Explains the effect as distraction; viewers watch the contact and now perceive it as correct rather than missing it."},{"text":"The added transient masks the frame-to-frame inconsistency in the generated contact, because an abrupt sound reduces temporal acuity in vision for a short window afterwards.","correct":false,"tempting":"Borrows a real masking phenomenon from audition and applies it to vision, where no such suppression accounts for the effect."},{"text":"The sound establishes the material of both objects, and once a viewer knows what the objects are made of they interpret the motion as consistent with that weight.","correct":false,"tempting":"Material identification is part of what the sound conveys, and the perceptual integration happens below the level of any inference about materials."},{"text":"Audio and visual information are combined involuntarily, and where the picture is briefly ambiguous the sound determines what is perceived.","correct":true}]},"prev":{"slug":"music-and-its-licence","title":"Music you could defend if somebody asked","url":"https://addaly.com/learn/ai-video/music-and-its-licence"},"next":{"slug":"models-that-generate-their-own-audio","title":"When the model makes the sound too","url":"https://addaly.com/learn/ai-video/models-that-generate-their-own-audio"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The viewer believes the sound', in Video with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-video/sound-design-fixes-bad-physics (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The viewer believes the sound","courseTitle":"Video with AI","module":{"index":5,"slug":"sound-and-speech","title":"The half of video that is not picture"},"answer":null,"keyIdea":"When sound and picture disagree, perception leans on the sound, which is why a well-synced impact makes a physically wrong collision read as correct and why sound design is a repair tool as well as a finishing one.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"left":{"title":"Repaired by a well-placed sound","items":["A glass breaking into the wrong number of pieces, at speed","Footsteps whose weight does not match the body above them","A hand meeting an object, a door closing, a cup set down","An event kept off screen and carried by a reaction shot"]},"type":"compare","right":{"title":"Repaired by nothing you can add","items":["A hand with six fingers","A face that changes identity mid-shot","Text melting across a sign","The same impact played at forty per cent speed"]},"title":"What a sound on the right frame repairs","caption":"Sound resolves ambiguity; it does not overwrite something clearly seen. Slow motion removes the ambiguity by giving the viewer time to check, which is the mechanism behind the rule never to put generated physics in slow motion."},"alt":"Figure: What a sound on the right frame repairs. Repaired by a well-placed sound: A glass breaking into the wrong number of pieces, at speed; Footsteps whose weight does not match the body above them; A hand meeting an object, a door closing, a cup set down; An event kept off screen and carried by a reaction shot. Repaired by nothing you can add: A hand with six fingers; A face that changes identity mid-shot; Text melting across a sign; The same impact played at forty per cent speed. Sound resolves ambiguity; it does not overwrite something clearly seen. Slow motion removes the ambiguity by giving the viewer time to check, which is the mechanism behind the rule never to put generated physics in slow motion."}},"blocks":[{"t":"h2","id":"audio-wins-arguments-with-vision","runs":[{"text":"Audio wins arguments with vision"}]},{"t":"p","runs":[{"text":"This is not a metaphor. It is a measurable perceptual effect."}]},{"t":"p","runs":[{"text":"The McGurk effect is the famous demonstration: a video of a mouth saying one syllable dubbed with a different syllable is heard as a third thing that neither source contains. Vision and hearing are being combined, and the combination is involuntary: knowing about the effect does not switch it off."}]},{"t":"p","runs":[{"text":"Related findings run through the whole perception literature. Sound changes the perceived timing of visual events. Sound changes the perceived force of an impact. A single flash accompanied by two beeps is often perceived as two flashes."}]},{"t":"p","runs":[{"text":"For generative video this has a direct, exploitable consequence: "},{"text":"a sound can repair a picture","b":true},{"text":"."}]},{"t":"h2","id":"where-this-pays","runs":[{"text":"Where this pays"}]},{"t":"p","runs":[{"text":"Recall the list of things the model cannot do. Several of them are substantially fixable with audio."}]},{"t":"p","runs":[{"text":"Impacts and collisions.","b":true},{"text":" A glass hitting a floor and breaking into the wrong number of pieces is one of the model’s weaknesses. Put a real, sharp, correctly-timed breaking-glass sound on the exact frame of contact and the shot reads. The eye had a moment of ambiguity and the ear resolved it."}]},{"t":"p","runs":[{"text":"Footsteps.","b":true},{"text":" Generated walking often has slightly wrong weight: the foot lands and the body does not respond correctly. Footsteps recorded on the right surface, placed on the contact frames, supply the weight the picture lacks."}]},{"t":"p","runs":[{"text":"Contact events generally.","b":true},{"text":" A hand touching an object, a door closing, a cup set down. These are exactly the moments the model handles worst, and they are also the moments that carry the clearest sounds."}]},{"t":"p","runs":[{"text":"Off-screen causes.","b":true},{"text":" The strongest version. A sound implies an event the picture never had to show. A car crash heard over a reaction shot is a car crash, and you generated nothing."}]},{"t":"h2","id":"the-technique-precisely","runs":[{"text":"The technique, precisely"}]},{"t":"p","runs":[{"text":"Three things and they are all about precision."}]},{"t":"ol","start":1,"items":[[{"text":"Find the exact frame of contact.","b":true},{"text":" Step through the clip frame by frame. The contact frame is usually one or two earlier than you think, because you are reacting to the frame where the change is obvious rather than where it began."}],[{"text":"Place the transient on that frame.","b":true},{"text":" The transient is the sharp attack at the start of the sound, which is not necessarily the start of the file. Zoom into the waveform and align the spike, not the edge."}],[{"text":"Match the sound’s character to the picture.","b":true},{"text":" A light object needs a light sound. Nothing exposes a fake more than a heavy impact on a small event, and this is worth watching for, because stock libraries are full of exaggerated sounds designed for trailers."}]]},{"t":"p","runs":[{"text":"Two frames late reads as wrong. On the frame, it disappears into the picture and does its work."}]},{"t":"h2","id":"what-sound-cannot-fix","runs":[{"text":"What sound cannot fix"}]},{"t":"p","runs":[{"text":"Honesty about the limit. Sound resolves ambiguity; it does not overwrite something clearly seen."}]},{"t":"ul","items":[[{"text":"A hand with six fingers is not fixed by any sound."}],[{"text":"A face that changes identity mid-shot is not fixed by any sound."}],[{"text":"Text that melts is not fixed by any sound."}],[{"text":"Slow motion defeats the whole technique. The audience has been given time to look, the ambiguity is resolved visually, and no audio cue competes with that. This is the mechanism behind the earlier advice never to put fake physics in slow motion."}]]},{"t":"p","runs":[{"text":"The rule: sound repairs fast, ambiguous, brief events. It does nothing for sustained, clearly visible errors."}]},{"t":"h2","id":"designing-the-cut-around-this","runs":[{"text":"Designing the cut around this"}]},{"t":"p","runs":[{"text":"Once you know sound carries impacts, you can build shots that rely on it:"}]},{"t":"ul","items":[[{"text":"Cut away "},{"text":"at","i":true},{"text":" the contact and let the sound continue over the next shot. The audience assembles an event they never saw. This is the oldest trick in the medium and it works perfectly here."}],[{"text":"Put difficult physics off-screen entirely and cover it with sound plus a reaction."}],[{"text":"Let a sound start before its picture (an L-cut) so the next shot arrives already explained."}]]},{"t":"p","runs":[{"text":"These are standard editing moves. They are worth restating here because in generative work they are not merely stylish, they are how you get shots you could not otherwise generate."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"Everything: Audacity to trim and shape, Freesound or your own phone for material, and any editor with frame-accurate placement. Resolve’s free tier shows waveforms at frame level. Kdenlive and Shotcut do too."}]},{"t":"p","runs":[{"text":"The skill is precision, and precision costs attention rather than money."}]},{"t":"rule"},{"t":"p","runs":[{"text":"Today: take a generated clip with a contact event that looks slightly wrong. Find the contact frame, place one real sound on it, and watch it again."}]}],"check":{"question":[{"text":"A generated shot of a mug being set down on a table looks slightly wrong: the contact reads as weightless. Adding a recorded mug-on-wood sound on the contact frame fixes the impression. Why does this work?"}],"options":[{"text":[{"text":"The sound draws attention away from the contact moment, so the viewer’s eye is elsewhere when the error occurs."}],"correct":false,"tempting":[{"text":"Explains the effect as distraction; viewers watch the contact and now perceive it as correct rather than missing it."}]},{"text":[{"text":"The added transient masks the frame-to-frame inconsistency in the generated contact, because an abrupt sound reduces temporal acuity in vision for a short window afterwards."}],"correct":false,"tempting":[{"text":"Borrows a real masking phenomenon from audition and applies it to vision, where no such suppression accounts for the effect."}]},{"text":[{"text":"The sound establishes the material of both objects, and once a viewer knows what the objects are made of they interpret the motion as consistent with that weight."}],"correct":false,"tempting":[{"text":"Material identification is part of what the sound conveys, and the perceptual integration happens below the level of any inference about materials."}]},{"text":[{"text":"Audio and visual information are combined involuntarily, and where the picture is briefly ambiguous the sound determines what is perceived."}],"correct":true,"tempting":null}],"order":[2,0,3,1],"why":null},"prev":{"slug":"music-and-its-licence","title":"Music you could defend if somebody asked","minutes":9},"next":{"slug":"models-that-generate-their-own-audio","title":"When the model makes the sound too","minutes":8}}}