{"course":{"slug":"ai-video","title":"Video with AI","url":"https://addaly.com/learn/ai-video","level":"intermediate","track":"create"},"lesson":{"slug":"performance-transfer","title":"Acting is timing, and timing has to come from somewhere","url":"https://addaly.com/learn/ai-video/performance-transfer","position":42,"of":66,"minutes":9,"answer":"Performance transfer drives a generated face from a recorded one, capturing the pauses, emphasis and eyebrow movement that carry meaning.","keyIdea":"Performance transfer drives a generated face from a recorded one, capturing the pauses, emphasis and eyebrow movement that carry meaning, which is why it beats lip-sync on anything with emotional content and why the person in front of the webcam matters more than the model."},"body":{"format":"markdown","text":"## What it does differently\n\nInstead of deriving mouth shapes from audio, performance transfer derives the whole facial performance from a video of a person acting. You record yourself on a webcam delivering the line; the model maps your expression, head movement, blinks and eye direction onto a generated character.\n\nRunway's Act-One is the best-known example of the class, and similar features exist elsewhere under other names.\n\nThe difference in output quality is not subtle, and the reason is not technical sophistication. It is that **delivery is most of acting**, and no phoneme model invents delivery.\n\n## What delivery actually consists of\n\nWatch somebody say a sentence they mean. The mouth is doing the least interesting part.\n\n- **Pauses.** Where somebody stops, and for how long. A quarter-second before a word changes what the word means.\n- **Emphasis.** Which syllable gets weight, and the tiny head movement that accompanies it.\n- **Eyebrows.** Raised on a question, drawn together on difficulty, lifted on surprise. These carry as much meaning as the words.\n- **Blinks.** People blink at clause boundaries and at moments of thought. Regular mechanical blinking reads as wrong.\n- **Eye direction.** Looking away while thinking, back on the point being made.\n- **Micro-stillness.** Genuine listening is very still. Generated faces tend to fidget.\n\nA lip-sync model produces none of these. A performance transfer produces all of them, because a person did them.\n\n## Which means the webcam is the important part\n\nThis is the unintuitive conclusion and the practically useful one. The quality of your talking-head shot depends mostly on how well the person in front of the webcam performs, and only secondarily on the model.\n\nSo:\n\n- **Perform the line properly.** Read it aloud several times first. Know what it means. Decide where the pauses are.\n- **Sit still and frontal**, well lit, camera at eye level. The model tracks your face, so give it a face to track: even light, no strong shadow across the features, no backlight.\n- **Do several takes** and pick the one where the delivery is right. This is free: it is a webcam recording, not a generation.\n- **Overplay very slightly.** Expression loses a little amplitude in transfer, as it does in animation generally.\n\nIf you cannot act, find somebody who can. A friend who does theatre will produce a better result than any amount of prompt engineering, and they will do it in ten minutes.\n\n## The limits\n\n- **Proportions transfer, and they should not.** A driver with a long face drives the target toward a long face. Pick a driver whose head shape is roughly compatible, or expect the character to shift toward you.\n- **Head rotation range is limited.** Large turns lose tracking. Keep within perhaps thirty degrees of frontal.\n- **Occlusion breaks it.** Hands near the face, hair falling across it, glasses catching the light.\n- **It is still a generated face.** Skin behaves slightly wrongly in motion, and long takes drift like everything else. Cut.\n- **Duration limits apply** exactly as they do to any generation.\n\n## Where it fits in a workflow\n\nA talking-head sequence that works usually looks like this:\n\n1. Record the audio properly, as a separate clean track: a phone in a quiet room beats a webcam microphone every time.\n2. Record the performance on the webcam, listening to or delivering the same line.\n3. Transfer the performance onto the character.\n4. Replace the webcam audio with the clean recording in the edit, aligned to the performance.\n5. Cut every few seconds, using cutaways and reaction shots so no single generation has to run long.\n\nStep four is the one people skip, and it is the difference between something that sounds like a video call and something that sounds produced.\n\n## The consent question is unchanged\n\nPerformance transfer onto a *generated* character is your own face driving a fictional one, which is straightforward. Performance transfer onto a *real person's* likeness is synthesising that person saying something they did not say, which requires their written, specific permission and a visible label on the output.\n\nThe technique does not change the rule and the quality of the result does not change it either. If anything, a better result raises the stakes, because a convincing synthetic performance is exactly what the disclosure obligations exist for.\n\n## The free path\n\nPerformance transfer is currently mostly a hosted, paid feature, and that is an honest limitation rather than something to talk around. The free approximations: open lip-sync plus a separately generated head-motion pass; pose and face-landmark conditioning in ComfyUI, which is fiddlier and does work; or the oldest answer, which is to film a real person and composite them, covered earlier in the course.\n\nRecording the audio well, which is half the result, is free on any phone.\n\n---\n\nToday: record yourself saying one line three ways: flat, with a pause before the last word, and with an eyebrow. Watch the three back. That difference is what transfer captures and lip-sync cannot."},"check":{"question":"Two teams produce a talking-head shot of the same generated character saying the same line. One uses audio-driven lip-sync with excellent audio; the other uses performance transfer from an amateur webcam recording. The second reads as more convincing. Why?","options":[{"text":"Performance transfer operates at higher resolution than lip-sync crops, so the second result has visibly better detail around the mouth.","correct":false,"tempting":"Names a real resolution difference in many implementations, which would not explain why the performance reads as more convincing rather than merely sharper."},{"text":"Audio-driven models introduce a small latency between sound and mouth movement, and viewers are extremely sensitive to audio-visual desynchronisation.","correct":false,"tempting":"Sync offset is a genuine failure mode, and the premise specifies excellent audio with correct timing."},{"text":"Lip-sync reconstructs only the lower face, so the upper face remains at whatever expression the source frame had, which viewers interpret as emotional flatness rather than as a technical limitation of the crop.","correct":false,"tempting":"Describes the mechanism accurately and stops one step short: the question is what the second method supplies, not only what the first omits."},{"text":"The pauses, emphasis and eyebrow movement that carry a line's meaning came from a person, and audio alone contains no instruction for them.","correct":true}]},"prev":{"slug":"lip-sync-mechanics","title":"What the model is actually reshaping","url":"https://addaly.com/learn/ai-video/lip-sync-mechanics"},"next":{"slug":"voice-cloning-and-synthetic-speech","title":"Synthetic voice, and where it stops being good enough","url":"https://addaly.com/learn/ai-video/voice-cloning-and-synthetic-speech"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Acting is timing, and timing has to come from somewhere', in Video with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-video/performance-transfer (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Acting is timing, and timing has to come from somewhere","courseTitle":"Video with AI","module":{"index":5,"slug":"sound-and-speech","title":"The half of video that is not picture"},"answer":null,"keyIdea":"Performance transfer drives a generated face from a recorded one, capturing the pauses, emphasis and eyebrow movement that carry meaning, which is why it beats lip-sync on anything with emotional content and why the person in front of the webcam matters more than the model.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-it-does-differently","runs":[{"text":"What it does differently"}]},{"t":"p","runs":[{"text":"Instead of deriving mouth shapes from audio, performance transfer derives the whole facial performance from a video of a person acting. You record yourself on a webcam delivering the line; the model maps your expression, head movement, blinks and eye direction onto a generated character."}]},{"t":"p","runs":[{"text":"Runway’s Act-One is the best-known example of the class, and similar features exist elsewhere under other names."}]},{"t":"p","runs":[{"text":"The difference in output quality is not subtle, and the reason is not technical sophistication. It is that "},{"text":"delivery is most of acting","b":true},{"text":", and no phoneme model invents delivery."}]},{"t":"h2","id":"what-delivery-actually-consists-of","runs":[{"text":"What delivery actually consists of"}]},{"t":"p","runs":[{"text":"Watch somebody say a sentence they mean. The mouth is doing the least interesting part."}]},{"t":"ul","items":[[{"text":"Pauses.","b":true},{"text":" Where somebody stops, and for how long. A quarter-second before a word changes what the word means."}],[{"text":"Emphasis.","b":true},{"text":" Which syllable gets weight, and the tiny head movement that accompanies it."}],[{"text":"Eyebrows.","b":true},{"text":" Raised on a question, drawn together on difficulty, lifted on surprise. These carry as much meaning as the words."}],[{"text":"Blinks.","b":true},{"text":" People blink at clause boundaries and at moments of thought. Regular mechanical blinking reads as wrong."}],[{"text":"Eye direction.","b":true},{"text":" Looking away while thinking, back on the point being made."}],[{"text":"Micro-stillness.","b":true},{"text":" Genuine listening is very still. Generated faces tend to fidget."}]]},{"t":"p","runs":[{"text":"A lip-sync model produces none of these. A performance transfer produces all of them, because a person did them."}]},{"t":"h2","id":"which-means-the-webcam-is-the-important-part","runs":[{"text":"Which means the webcam is the important part"}]},{"t":"p","runs":[{"text":"This is the unintuitive conclusion and the practically useful one. The quality of your talking-head shot depends mostly on how well the person in front of the webcam performs, and only secondarily on the model."}]},{"t":"p","runs":[{"text":"So:"}]},{"t":"ul","items":[[{"text":"Perform the line properly.","b":true},{"text":" Read it aloud several times first. Know what it means. Decide where the pauses are."}],[{"text":"Sit still and frontal","b":true},{"text":", well lit, camera at eye level. The model tracks your face, so give it a face to track: even light, no strong shadow across the features, no backlight."}],[{"text":"Do several takes","b":true},{"text":" and pick the one where the delivery is right. This is free: it is a webcam recording, not a generation."}],[{"text":"Overplay very slightly.","b":true},{"text":" Expression loses a little amplitude in transfer, as it does in animation generally."}]]},{"t":"p","runs":[{"text":"If you cannot act, find somebody who can. A friend who does theatre will produce a better result than any amount of prompt engineering, and they will do it in ten minutes."}]},{"t":"h2","id":"the-limits","runs":[{"text":"The limits"}],"role":"limit"},{"t":"ul","items":[[{"text":"Proportions transfer, and they should not.","b":true},{"text":" A driver with a long face drives the target toward a long face. Pick a driver whose head shape is roughly compatible, or expect the character to shift toward you."}],[{"text":"Head rotation range is limited.","b":true},{"text":" Large turns lose tracking. Keep within perhaps thirty degrees of frontal."}],[{"text":"Occlusion breaks it.","b":true},{"text":" Hands near the face, hair falling across it, glasses catching the light."}],[{"text":"It is still a generated face.","b":true},{"text":" Skin behaves slightly wrongly in motion, and long takes drift like everything else. Cut."}],[{"text":"Duration limits apply","b":true},{"text":" exactly as they do to any generation."}]]},{"t":"h2","id":"where-it-fits-in-a-workflow","runs":[{"text":"Where it fits in a workflow"}]},{"t":"p","runs":[{"text":"A talking-head sequence that works usually looks like this:"}]},{"t":"ol","start":1,"items":[[{"text":"Record the audio properly, as a separate clean track: a phone in a quiet room beats a webcam microphone every time."}],[{"text":"Record the performance on the webcam, listening to or delivering the same line."}],[{"text":"Transfer the performance onto the character."}],[{"text":"Replace the webcam audio with the clean recording in the edit, aligned to the performance."}],[{"text":"Cut every few seconds, using cutaways and reaction shots so no single generation has to run long."}]]},{"t":"p","runs":[{"text":"Step four is the one people skip, and it is the difference between something that sounds like a video call and something that sounds produced."}]},{"t":"h2","id":"the-consent-question-is-unchanged","runs":[{"text":"The consent question is unchanged"}]},{"t":"p","runs":[{"text":"Performance transfer onto a "},{"text":"generated","i":true},{"text":" character is your own face driving a fictional one, which is straightforward. Performance transfer onto a "},{"text":"real person’s","i":true},{"text":" likeness is synthesising that person saying something they did not say, which requires their written, specific permission and a visible label on the output."}]},{"t":"p","runs":[{"text":"The technique does not change the rule and the quality of the result does not change it either. If anything, a better result raises the stakes, because a convincing synthetic performance is exactly what the disclosure obligations exist for."}]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"Performance transfer is currently mostly a hosted, paid feature, and that is an honest limitation rather than something to talk around. The free approximations: open lip-sync plus a separately generated head-motion pass; pose and face-landmark conditioning in ComfyUI, which is fiddlier and does work; or the oldest answer, which is to film a real person and composite them, covered earlier in the course."}]},{"t":"p","runs":[{"text":"Recording the audio well, which is half the result, is free on any phone."}]},{"t":"rule"},{"t":"p","runs":[{"text":"Today: record yourself saying one line three ways: flat, with a pause before the last word, and with an eyebrow. Watch the three back. That difference is what transfer captures and lip-sync cannot."}]}],"check":{"question":[{"text":"Two teams produce a talking-head shot of the same generated character saying the same line. One uses audio-driven lip-sync with excellent audio; the other uses performance transfer from an amateur webcam recording. The second reads as more convincing. Why?"}],"options":[{"text":[{"text":"Performance transfer operates at higher resolution than lip-sync crops, so the second result has visibly better detail around the mouth."}],"correct":false,"tempting":[{"text":"Names a real resolution difference in many implementations, which would not explain why the performance reads as more convincing rather than merely sharper."}]},{"text":[{"text":"Audio-driven models introduce a small latency between sound and mouth movement, and viewers are extremely sensitive to audio-visual desynchronisation."}],"correct":false,"tempting":[{"text":"Sync offset is a genuine failure mode, and the premise specifies excellent audio with correct timing."}]},{"text":[{"text":"Lip-sync reconstructs only the lower face, so the upper face remains at whatever expression the source frame had, which viewers interpret as emotional flatness rather than as a technical limitation of the crop."}],"correct":false,"tempting":[{"text":"Describes the mechanism accurately and stops one step short: the question is what the second method supplies, not only what the first omits."}]},{"text":[{"text":"The pauses, emphasis and eyebrow movement that carry a line’s meaning came from a person, and audio alone contains no instruction for them."}],"correct":true,"tempting":null}],"order":[2,3,0,1],"why":null},"prev":{"slug":"lip-sync-mechanics","title":"What the model is actually reshaping","minutes":9},"next":{"slug":"voice-cloning-and-synthetic-speech","title":"Synthetic voice, and where it stops being good enough","minutes":9}}}