{"course":{"slug":"motion-and-audio","title":"Motion, Sound and Music","url":"https://addaly.com/learn/motion-and-audio","level":"intermediate","track":"create"},"lesson":{"slug":"captions-and-word-error-rate","title":"Five per cent, and which five per cent","url":"https://addaly.com/learn/motion-and-audio/captions-and-word-error-rate","position":67,"of":72,"minutes":11,"answer":"Word error rate is a uniform average over errors that are not uniformly distributed (they cluster on names, numbers and technical terms) and a generative recogniser can produce fluent sentences during silence.","keyIdea":"Word error rate is a uniform average over errors that are not uniformly distributed (they cluster on names, numbers and technical terms) and a generative recogniser can produce fluent sentences during silence, so no automatic transcript is publishable unread."},"body":{"format":"markdown","text":"## What a WER figure is actually telling you\n\nAutomatic speech recognition is scored with **word error rate**:\n\n```\nWER = (substitutions + deletions + insertions) / words in the reference\n```\n\nA 5 per cent WER means one word in twenty is wrong. On a 1,000-word video that is fifty errors. Stated as a percentage it sounds like a rounding error; stated as fifty wrong words in ten minutes it sounds like what it is.\n\nThe figure also understates the damage, for a structural reason. Errors are not spread evenly. They concentrate on **proper nouns, technical terms, numbers and rare words**: precisely the content words a viewer is relying on the captions for. A transcript can be 95 per cent accurate and get every name in it wrong, and the 5 per cent that failed is the 5 per cent that mattered.\n\n## Where accuracy actually goes\n\n- **Overlapping speech.** Two people talking at once defeats most systems, and this is most of a real conversation.\n- **Accents and dialects** under-represented in training data. Published research has repeatedly found substantially higher error rates for some speaker groups than others on the same systems; this is a known and documented disparity, not an anomaly.\n- **Background noise and music.**\n- **Domain vocabulary.** Medical, legal, technical and organisational terms.\n- **Code-switching.** Moving between languages mid-sentence, which is ordinary speech for a large part of the world and poorly handled.\n\nMany systems accept a vocabulary or prompt hint. Supplying a list of the names and terms in your video before transcription is the single cheapest accuracy improvement available.\n\n## The hallucination problem\n\nWhisper, the most widely used open model, has a documented failure mode that is different in kind from ordinary errors: during silence, music, or non-speech audio, it can generate fluent text that nobody said. Researchers examining transcriptions have found invented sentences, and the phrase \"Thank you for watching\" appearing at the ends of clips is a well-known artefact of its training data.\n\nThis matters because the output is confident and well-formed. A conventional recogniser fails visibly, producing garbage you can see. A generative one fails invisibly, producing a plausible sentence.\n\nMitigations that help: setting a voice-activity detector to skip non-speech, using `condition_on_previous_text=False` to stop an error propagating forward, and checking any segment of silence in the output. None of them removes the need to read the transcript.\n\n## Captions as a craft, not an export\n\nAn auto-generated caption track is a starting point. The conventions that make captions readable are well established:\n\n> Figure: The conventions that make a caption readable. Layers, top to bottom: 32 to 42 characters a line (two lines on screen at most); 17 to 20 characters a second (about 160 to 180 words a minute); One second minimum (six or seven at the outside); Break at a grammatical boundary (not where the width happened to run out); Clear of the mouth (and of any burned-in text); Name the speaker when unclear (and describe sound, for SDH). An automatic track is a starting point, not an export. Errors concentrate on names, numbers and technical terms, the words a viewer is relying on the captions for, so a transcript can be 95 per cent accurate and get every name in it wrong. Reading a ten-minute transcript takes about four minutes.\n\n- **32 to 42 characters per line**, at most two lines on screen.\n- **Reading rate** around 160 to 180 words per minute for adult viewers, often expressed as 17 to 20 characters per second. Faster than that and people are still reading when the caption changes.\n- **Minimum one second** on screen, even for one word; **maximum about six or seven seconds**.\n- **Break lines at grammatical boundaries** (after a clause, before a preposition) not wherever the width runs out. A line broken mid-phrase measurably slows reading.\n- **Do not cover the speaker's mouth** or burned-in text.\n- **Identify speakers** when it is not obvious, and describe significant non-speech sound for SDH.\n\n**Subtitles** assume the viewer can hear and translate or transcribe dialogue. **SDH** (subtitles for the deaf and hard of hearing) additionally carry speaker identification and sound effects. They are different deliverables.\n\n## Burned-in or sidecar\n\n**Sidecar files** (SRT, WebVTT, TTML) are separate and referenced by the player. They can be toggled off, restyled, searched, indexed by search engines, and translated. They are the correct default.\n\n**Burned-in captions** are pixels. They survive platforms that strip sidecar tracks, they cannot be turned off, they cannot be restyled by a viewer who needs larger text, and they are baked into every version of the file.\n\nSocial platforms where video autoplays muted are the real case for burning in, and many accept a sidecar as well. The defensible approach for those is both: burned-in for the feed, sidecar for accessibility.\n\n```bash\n# transcribe locally\nwhisper-ctranslate2 --model medium --language en --vad_filter True talk.mp4\n\n# attach as a soft track, no re-encode of the video\nffmpeg -i talk.mp4 -i talk.srt -c copy -c:s mov_text out.mp4\n\n# burn in, with styling\nffmpeg -i talk.mp4 -vf \"subtitles=talk.srt:force_style='FontSize=22,Outline=2'\" out.mp4\n```\n\n## The free path\n\n- **Whisper** and the faster **faster-whisper** / **whisper.cpp** implementations run locally on modest hardware, are free, and handle many languages.\n- **Subtitle Edit** is free and open source, and is a good caption editor with waveform view, timing tools and format conversion.\n- **Aegisub** for styled subtitles.\n- **ffmpeg** for muxing and burning in.\n- **DaVinci Resolve**'s free edition includes transcription and a subtitle track.\n\n## The limitation\n\nCaptions are a legal requirement in many contexts, and the requirements differ by country, by sector and by whether the content is broadcast, educational, governmental or commercial. Several jurisdictions have specific standards and enforcement histories. This is not something to resolve from a general course: check the rules that apply to your jurisdiction and your kind of publication, and take advice if the answer matters.\n\nThe technical limitation is simpler and is the one to hold on to: no automatic system is accurate enough to publish unread. Reading a ten-minute transcript takes about four minutes and catches the names, the numbers and the sentences nobody said."},"check":{"question":"A transcript produced by Whisper contains a well-formed sentence during a fifteen-second passage where nobody is speaking. Why is this failure different in kind from an ordinary recognition error?","options":[{"text":"It is a timing error rather than a recognition error: the model has assigned a correctly recognised sentence to the wrong timestamp, which a re-alignment pass will correct.","correct":false,"tempting":"Misalignment is a separate real problem; here the words were never spoken at any timestamp."},{"text":"The model is generative, so it fails by producing plausible text rather than visible garbage, and a reader has no signal that anything went wrong.","correct":true},{"text":"The error is an insertion rather than a substitution, and insertions are weighted more heavily in the word error rate calculation than other error types.","correct":false,"tempting":"All three error types are weighted equally in the standard formula; the significance is perceptual rather than arithmetic."},{"text":"Silence contains no acoustic features, so the model falls back to its language model alone, and a pure language-model output is by definition unrelated to the audio and therefore excluded from the word error rate denominator.","correct":false,"tempting":"Describes a plausible mechanism, but such output is scored as an ordinary insertion and counts against the transcript."}]},"prev":{"slug":"a-cloned-voice-needs-consent","title":"A cloned voice needs a yes you can point to","url":"https://addaly.com/learn/motion-and-audio/a-cloned-voice-needs-consent"},"next":{"slug":"separating-a-mix","title":"Taking a mixture apart","url":"https://addaly.com/learn/motion-and-audio/separating-a-mix"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Five per cent, and which five per cent', in Motion, Sound and Music. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/motion-and-audio/captions-and-word-error-rate (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Five per cent, and which five per cent","courseTitle":"Motion, Sound and Music","module":{"index":8,"slug":"machines-that-make-sound-and-motion","title":"Machines that make sound and motion"},"answer":null,"keyIdea":"Word error rate is a uniform average over errors that are not uniformly distributed (they cluster on names, numbers and technical terms) and a generative recogniser can produce fluent sentences during silence, so no automatic transcript is publishable unread.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"The conventions that make a caption readable","layers":[{"label":"32 to 42 characters a line","detail":"two lines on screen at most"},{"label":"17 to 20 characters a second","detail":"about 160 to 180 words a minute"},{"label":"One second minimum","detail":"six or seven at the outside"},{"label":"Break at a grammatical boundary","detail":"not where the width happened to run out"},{"label":"Clear of the mouth","detail":"and of any burned-in text"},{"label":"Name the speaker when unclear","detail":"and describe sound, for SDH"}],"caption":"An automatic track is a starting point, not an export. Errors concentrate on names, numbers and technical terms, the words a viewer is relying on the captions for, so a transcript can be 95 per cent accurate and get every name in it wrong. Reading a ten-minute transcript takes about four minutes."},"alt":"Figure: The conventions that make a caption readable. Layers, top to bottom: 32 to 42 characters a line (two lines on screen at most); 17 to 20 characters a second (about 160 to 180 words a minute); One second minimum (six or seven at the outside); Break at a grammatical boundary (not where the width happened to run out); Clear of the mouth (and of any burned-in text); Name the speaker when unclear (and describe sound, for SDH). An automatic track is a starting point, not an export. Errors concentrate on names, numbers and technical terms, the words a viewer is relying on the captions for, so a transcript can be 95 per cent accurate and get every name in it wrong. Reading a ten-minute transcript takes about four minutes."}},"blocks":[{"t":"h2","id":"what-a-wer-figure-is-actually-telling-you","runs":[{"text":"What a WER figure is actually telling you"}]},{"t":"p","runs":[{"text":"Automatic speech recognition is scored with "},{"text":"word error rate","b":true},{"text":":"}]},{"t":"code","lang":"","label":"Text","text":"WER = (substitutions + deletions + insertions) / words in the reference"},{"t":"p","runs":[{"text":"A 5 per cent WER means one word in twenty is wrong. On a 1,000-word video that is fifty errors. Stated as a percentage it sounds like a rounding error; stated as fifty wrong words in ten minutes it sounds like what it is."}]},{"t":"p","runs":[{"text":"The figure also understates the damage, for a structural reason. Errors are not spread evenly. They concentrate on "},{"text":"proper nouns, technical terms, numbers and rare words","b":true},{"text":": precisely the content words a viewer is relying on the captions for. A transcript can be 95 per cent accurate and get every name in it wrong, and the 5 per cent that failed is the 5 per cent that mattered."}]},{"t":"h2","id":"where-accuracy-actually-goes","runs":[{"text":"Where accuracy actually goes"}]},{"t":"ul","items":[[{"text":"Overlapping speech.","b":true},{"text":" Two people talking at once defeats most systems, and this is most of a real conversation."}],[{"text":"Accents and dialects","b":true},{"text":" under-represented in training data. Published research has repeatedly found substantially higher error rates for some speaker groups than others on the same systems; this is a known and documented disparity, not an anomaly."}],[{"text":"Background noise and music.","b":true}],[{"text":"Domain vocabulary.","b":true},{"text":" Medical, legal, technical and organisational terms."}],[{"text":"Code-switching.","b":true},{"text":" Moving between languages mid-sentence, which is ordinary speech for a large part of the world and poorly handled."}]]},{"t":"p","runs":[{"text":"Many systems accept a vocabulary or prompt hint. Supplying a list of the names and terms in your video before transcription is the single cheapest accuracy improvement available."}]},{"t":"h2","id":"the-hallucination-problem","runs":[{"text":"The hallucination problem"}]},{"t":"p","runs":[{"text":"Whisper, the most widely used open model, has a documented failure mode that is different in kind from ordinary errors: during silence, music, or non-speech audio, it can generate fluent text that nobody said. Researchers examining transcriptions have found invented sentences, and the phrase “Thank you for watching” appearing at the ends of clips is a well-known artefact of its training data."}]},{"t":"p","runs":[{"text":"This matters because the output is confident and well-formed. A conventional recogniser fails visibly, producing garbage you can see. A generative one fails invisibly, producing a plausible sentence."}]},{"t":"p","runs":[{"text":"Mitigations that help: setting a voice-activity detector to skip non-speech, using "},{"text":"condition_on_previous_text=False","code":true},{"text":" to stop an error propagating forward, and checking any segment of silence in the output. None of them removes the need to read the transcript."}]},{"t":"h2","id":"captions-as-a-craft-not-an-export","runs":[{"text":"Captions as a craft, not an export"}]},{"t":"p","runs":[{"text":"An auto-generated caption track is a starting point. The conventions that make captions readable are well established:"}]},{"t":"ul","items":[[{"text":"32 to 42 characters per line","b":true},{"text":", at most two lines on screen."}],[{"text":"Reading rate","b":true},{"text":" around 160 to 180 words per minute for adult viewers, often expressed as 17 to 20 characters per second. Faster than that and people are still reading when the caption changes."}],[{"text":"Minimum one second","b":true},{"text":" on screen, even for one word; "},{"text":"maximum about six or seven seconds","b":true},{"text":"."}],[{"text":"Break lines at grammatical boundaries","b":true},{"text":" (after a clause, before a preposition) not wherever the width runs out. A line broken mid-phrase measurably slows reading."}],[{"text":"Do not cover the speaker’s mouth","b":true},{"text":" or burned-in text."}],[{"text":"Identify speakers","b":true},{"text":" when it is not obvious, and describe significant non-speech sound for SDH."}]]},{"t":"p","runs":[{"text":"Subtitles","b":true},{"text":" assume the viewer can hear and translate or transcribe dialogue. "},{"text":"SDH","b":true},{"text":" (subtitles for the deaf and hard of hearing) additionally carry speaker identification and sound effects. They are different deliverables."}]},{"t":"h2","id":"burned-in-or-sidecar","runs":[{"text":"Burned-in or sidecar"}]},{"t":"p","runs":[{"text":"Sidecar files","b":true},{"text":" (SRT, WebVTT, TTML) are separate and referenced by the player. They can be toggled off, restyled, searched, indexed by search engines, and translated. They are the correct default."}]},{"t":"p","runs":[{"text":"Burned-in captions","b":true},{"text":" are pixels. They survive platforms that strip sidecar tracks, they cannot be turned off, they cannot be restyled by a viewer who needs larger text, and they are baked into every version of the file."}]},{"t":"p","runs":[{"text":"Social platforms where video autoplays muted are the real case for burning in, and many accept a sidecar as well. The defensible approach for those is both: burned-in for the feed, sidecar for accessibility."}]},{"t":"code","lang":"bash","label":"Bash","text":"# transcribe locally\nwhisper-ctranslate2 --model medium --language en --vad_filter True talk.mp4\n\n# attach as a soft track, no re-encode of the video\nffmpeg -i talk.mp4 -i talk.srt -c copy -c:s mov_text out.mp4\n\n# burn in, with styling\nffmpeg -i talk.mp4 -vf \"subtitles=talk.srt:force_style='FontSize=22,Outline=2'\" out.mp4"},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"ul","items":[[{"text":"Whisper","b":true},{"text":" and the faster "},{"text":"faster-whisper","b":true},{"text":" / "},{"text":"whisper.cpp","b":true},{"text":" implementations run locally on modest hardware, are free, and handle many languages."}],[{"text":"Subtitle Edit","b":true},{"text":" is free and open source, and is a good caption editor with waveform view, timing tools and format conversion."}],[{"text":"Aegisub","b":true},{"text":" for styled subtitles."}],[{"text":"ffmpeg","b":true},{"text":" for muxing and burning in."}],[{"text":"DaVinci Resolve","b":true},{"text":"’s free edition includes transcription and a subtitle track."}]]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"Captions are a legal requirement in many contexts, and the requirements differ by country, by sector and by whether the content is broadcast, educational, governmental or commercial. Several jurisdictions have specific standards and enforcement histories. This is not something to resolve from a general course: check the rules that apply to your jurisdiction and your kind of publication, and take advice if the answer matters."}]},{"t":"p","runs":[{"text":"The technical limitation is simpler and is the one to hold on to: no automatic system is accurate enough to publish unread. Reading a ten-minute transcript takes about four minutes and catches the names, the numbers and the sentences nobody said."}]}],"check":{"question":[{"text":"A transcript produced by Whisper contains a well-formed sentence during a fifteen-second passage where nobody is speaking. Why is this failure different in kind from an ordinary recognition error?"}],"options":[{"text":[{"text":"It is a timing error rather than a recognition error: the model has assigned a correctly recognised sentence to the wrong timestamp, which a re-alignment pass will correct."}],"correct":false,"tempting":[{"text":"Misalignment is a separate real problem; here the words were never spoken at any timestamp."}]},{"text":[{"text":"The model is generative, so it fails by producing plausible text rather than visible garbage, and a reader has no signal that anything went wrong."}],"correct":true,"tempting":null},{"text":[{"text":"The error is an insertion rather than a substitution, and insertions are weighted more heavily in the word error rate calculation than other error types."}],"correct":false,"tempting":[{"text":"All three error types are weighted equally in the standard formula; the significance is perceptual rather than arithmetic."}]},{"text":[{"text":"Silence contains no acoustic features, so the model falls back to its language model alone, and a pure language-model output is by definition unrelated to the audio and therefore excluded from the word error rate denominator."}],"correct":false,"tempting":[{"text":"Describes a plausible mechanism, but such output is scored as an ordinary insertion and counts against the transcript."}]}],"order":[0,3,1,2],"why":null},"prev":{"slug":"a-cloned-voice-needs-consent","title":"A cloned voice needs a yes you can point to","minutes":11},"next":{"slug":"separating-a-mix","title":"Taking a mixture apart","minutes":10}}}