{"course":{"slug":"motion-and-audio","title":"Motion, Sound and Music","url":"https://addaly.com/learn/motion-and-audio","level":"intermediate","track":"create"},"lesson":{"slug":"tts-and-prosody","title":"What a text-to-speech system cannot read","url":"https://addaly.com/learn/motion-and-audio/tts-and-prosody","position":64,"of":72,"minutes":10,"answer":"Written text does not encode stress, and stress is meaning, so a synthesiser guesses prosody from statistical regularities and produces a convincing reading of the wrong sentence.","keyIdea":"Written text does not encode stress, and stress is meaning, so a synthesiser guesses prosody from statistical regularities and produces a convincing reading of the wrong sentence, which is why a pronunciation lexicon and SSML are quality control rather than decoration."},"body":{"format":"markdown","text":"## Two stages, and the second one is not where the problems are\n\nMost text-to-speech systems work in two parts. A first model turns text into an intermediate acoustic representation: usually a mel-spectrogram, a time-frequency picture of the intended sound. A second model, the **vocoder**, turns that picture into a waveform. Newer systems compress the two into one by predicting tokens of a learned audio codec directly.\n\nThe vocoder is largely a solved problem: modern ones produce clean, natural-sounding audio. Almost everything that makes synthesised speech sound wrong happens in the first stage, and it is not about the voice at all. It is about **prosody**: the rhythm, stress and intonation that carry meaning beyond the words.\n\n## What the text does not contain\n\nSay this sentence aloud seven times, stressing a different word each time:\n\n> I never said she stole my money.\n\nEach version means something different. *I* never said it; someone else did. I never *said* it; I implied it. She stole my *money*, not my car. The written sentence contains none of that. The stress is the meaning, and it is absent from the input.\n\nA TTS model has to guess, and it guesses from statistical regularities in its training data. On ordinary declarative prose it does well. On anything where the emphasis carries the point (a contrast, a correction, an aside, a list where one item is the surprising one) it produces a plausible reading of the wrong sentence.\n\n## The specific failures\n\n**Homographs.** Words spelled the same and pronounced differently, disambiguated only by meaning:\n\n- *read*: \"I read it yesterday\" against \"I will read it\"\n- *lead*: the metal against the verb\n- *live*: \"live broadcast\" against \"where they live\"\n- *bass*, *minute*, *wind*, *tear*, *close*, *record*, *desert*, *row*\n\nModels get these right most of the time from context and wrong in a way that stops a listener.\n\n**Numbers and dates.** \"1996\" is *nineteen ninety-six* as a year and *one thousand nine hundred and ninety-six* as a quantity. \"3/4\" is a fraction, a date, or a ratio. \"10-12\" is a range or a subtraction. Phone numbers, version numbers, prices and measurements all have conventions the text does not state.\n\n**Names and jargon.** Proper nouns are where the errors concentrate, and they are the words a listener is most likely to be listening for.\n\n**Acronyms.** Some are spelled out, some are pronounced as words, and there is no rule in the text: NASA is a word, FBI is letters, SQL is both depending on who you ask.\n\n**Long-form structure.** Most systems synthesise a sentence or a chunk at a time with no memory of the paragraph. So the emphasis does not build, a list does not fall at the end, and a long read has a flat, sentence-by-sentence sameness that listeners describe as tiring without being able to say why.\n\n## SSML: the controls that exist\n\nSpeech Synthesis Markup Language is the standard way of supplying what the text lacks. Support varies by engine, but the core is widespread:\n\n```xml\n\n The report was published in 1996.\n She never agreed to it.\n SQL is one option.\n The alternative is simpler.\n This part is an aside.\n\n```\n\nA pronunciation lexicon (a list of your names and terms with their phonetic spellings) is worth building once for any project with recurring vocabulary, and it removes the most damaging category of error permanently.\n\n## Practical technique\n\n- **Write for the ear.** Shorter sentences, one idea each. A clause that a human reader would handle with a subtle pitch move will defeat the model.\n- **Punctuate for timing.** A full stop produces a longer pause than a comma. Splitting a long sentence into two changes the delivery more reliably than any tag.\n- **Spell things out in the input.** Writing \"nineteen ninety-six\" is often quicker than a tag and always works.\n- **Listen to the whole thing.** Errors are local and a spot check will miss them.\n- **Chunk long reads at paragraph boundaries**, and check the joins for a level or timbre shift.\n\n## The free path\n\n- **Piper**: fast, local, neural, dozens of languages, runs on a Raspberry Pi. The sensible default for free self-hosted TTS.\n- **eSpeak NG**: formant synthesis, very robotic, tiny, over a hundred languages, and still valuable for languages no neural model covers.\n- **Coqui TTS**: a widely used open-source toolkit; note that the company behind it shut down, so the repository is no longer actively maintained even though the models and code remain available and usable. Check the project's state before building on it.\n- **Mimic 3**, **MaryTTS**: other open options.\n- **Whisper** is the counterpart on the recognition side, covered in a later lesson.\n\n## The limitation\n\nSynthesised speech is now good enough that the remaining errors are more dangerous than the old obvious ones. A robotic voice signalled clearly that this was a machine; a natural voice confidently mispronouncing a drug name or a person's surname does not signal anything, and a listener may not notice.\n\nThere is also no reliable way for the model to know what it got wrong. It does not have a confidence score for prosody, and it will read a sentence with exactly the wrong emphasis in a perfectly convincing voice. The only check is a person listening to the output against the intended meaning, which is unglamorous and is the actual quality control step."},"check":{"question":"A synthesised narration reads a list of seven product features in a uniform, slightly tiring way, although each sentence on its own sounds natural. What is the structural cause?","options":[{"text":"The vocoder is producing identical spectral characteristics for each sentence, because it has no conditioning on position within a document.","correct":false,"tempting":"The vocoder converts an acoustic representation to a waveform; the flatness originates in the stage that produced that representation."},{"text":"Most systems synthesise a sentence or chunk at a time with no memory of the paragraph, so emphasis cannot build and a list does not fall at the end.","correct":true},{"text":"The sentences are too similar in length and structure, and varying the sentence length would give the model enough signal to produce contour variation.","correct":false,"tempting":"Varying the writing helps in practice, but the absence of cross-sentence context is the cause rather than the uniformity of the input."},{"text":"The model has not been given an SSML prosody tag, and without an explicit rate or pitch instruction every synthesiser defaults to a fixed monotone contour derived from its training corpus's mean fundamental frequency.","correct":false,"tempting":"Systems produce varied contours within a sentence without any tags; the problem is scope, not a default monotone."}]},"prev":{"slug":"the-order-of-operations","title":"The chain, and why the order is not arbitrary","url":"https://addaly.com/learn/motion-and-audio/the-order-of-operations"},"next":{"slug":"how-a-voice-is-cloned","title":"The vector that is supposed to be you","url":"https://addaly.com/learn/motion-and-audio/how-a-voice-is-cloned"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What a text-to-speech system cannot read', in Motion, Sound and Music. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/motion-and-audio/tts-and-prosody (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What a text-to-speech system cannot read","courseTitle":"Motion, Sound and Music","module":{"index":8,"slug":"machines-that-make-sound-and-motion","title":"Machines that make sound and motion"},"answer":null,"keyIdea":"Written text does not encode stress, and stress is meaning, so a synthesiser guesses prosody from statistical regularities and produces a convincing reading of the wrong sentence, which is why a pronunciation lexicon and SSML are quality control rather than decoration.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-stages-and-the-second-one-is-not-where-the-problems-are","runs":[{"text":"Two stages, and the second one is not where the problems are"}]},{"t":"p","runs":[{"text":"Most text-to-speech systems work in two parts. A first model turns text into an intermediate acoustic representation: usually a mel-spectrogram, a time-frequency picture of the intended sound. A second model, the "},{"text":"vocoder","b":true},{"text":", turns that picture into a waveform. Newer systems compress the two into one by predicting tokens of a learned audio codec directly."}]},{"t":"p","runs":[{"text":"The vocoder is largely a solved problem: modern ones produce clean, natural-sounding audio. Almost everything that makes synthesised speech sound wrong happens in the first stage, and it is not about the voice at all. It is about "},{"text":"prosody","b":true},{"text":": the rhythm, stress and intonation that carry meaning beyond the words."}]},{"t":"h2","id":"what-the-text-does-not-contain","runs":[{"text":"What the text does not contain"}]},{"t":"p","runs":[{"text":"Say this sentence aloud seven times, stressing a different word each time:"}]},{"t":"quote","runs":[{"text":"I never said she stole my money."}]},{"t":"p","runs":[{"text":"Each version means something different. "},{"text":"I","i":true},{"text":" never said it; someone else did. I never "},{"text":"said","i":true},{"text":" it; I implied it. She stole my "},{"text":"money","i":true},{"text":", not my car. The written sentence contains none of that. The stress is the meaning, and it is absent from the input."}]},{"t":"p","runs":[{"text":"A TTS model has to guess, and it guesses from statistical regularities in its training data. On ordinary declarative prose it does well. On anything where the emphasis carries the point (a contrast, a correction, an aside, a list where one item is the surprising one) it produces a plausible reading of the wrong sentence."}]},{"t":"h2","id":"the-specific-failures","runs":[{"text":"The specific failures"}]},{"t":"p","runs":[{"text":"Homographs.","b":true},{"text":" Words spelled the same and pronounced differently, disambiguated only by meaning:"}]},{"t":"ul","items":[[{"text":"read","i":true},{"text":": “I read it yesterday” against “I will read it”"}],[{"text":"lead","i":true},{"text":": the metal against the verb"}],[{"text":"live","i":true},{"text":": “live broadcast” against “where they live”"}],[{"text":"bass","i":true},{"text":", "},{"text":"minute","i":true},{"text":", "},{"text":"wind","i":true},{"text":", "},{"text":"tear","i":true},{"text":", "},{"text":"close","i":true},{"text":", "},{"text":"record","i":true},{"text":", "},{"text":"desert","i":true},{"text":", "},{"text":"row","i":true}]]},{"t":"p","runs":[{"text":"Models get these right most of the time from context and wrong in a way that stops a listener."}]},{"t":"p","runs":[{"text":"Numbers and dates.","b":true},{"text":" “1996” is "},{"text":"nineteen ninety-six","i":true},{"text":" as a year and "},{"text":"one thousand nine hundred and ninety-six","i":true},{"text":" as a quantity. “3/4” is a fraction, a date, or a ratio. “10-12” is a range or a subtraction. Phone numbers, version numbers, prices and measurements all have conventions the text does not state."}]},{"t":"p","runs":[{"text":"Names and jargon.","b":true},{"text":" Proper nouns are where the errors concentrate, and they are the words a listener is most likely to be listening for."}]},{"t":"p","runs":[{"text":"Acronyms.","b":true},{"text":" Some are spelled out, some are pronounced as words, and there is no rule in the text: NASA is a word, FBI is letters, SQL is both depending on who you ask."}]},{"t":"p","runs":[{"text":"Long-form structure.","b":true},{"text":" Most systems synthesise a sentence or a chunk at a time with no memory of the paragraph. So the emphasis does not build, a list does not fall at the end, and a long read has a flat, sentence-by-sentence sameness that listeners describe as tiring without being able to say why."}]},{"t":"h2","id":"ssml-the-controls-that-exist","runs":[{"text":"SSML: the controls that exist"}]},{"t":"p","runs":[{"text":"Speech Synthesis Markup Language is the standard way of supplying what the text lacks. Support varies by engine, but the core is widespread:"}]},{"t":"code","lang":"xml","label":"XML","text":"\n The report was published in 1996.\n She never agreed to it.\n SQL is one option.\n The alternative is simpler.\n This part is an aside.\n"},{"t":"p","runs":[{"text":"A pronunciation lexicon (a list of your names and terms with their phonetic spellings) is worth building once for any project with recurring vocabulary, and it removes the most damaging category of error permanently."}]},{"t":"h2","id":"practical-technique","runs":[{"text":"Practical technique"}]},{"t":"ul","items":[[{"text":"Write for the ear.","b":true},{"text":" Shorter sentences, one idea each. A clause that a human reader would handle with a subtle pitch move will defeat the model."}],[{"text":"Punctuate for timing.","b":true},{"text":" A full stop produces a longer pause than a comma. Splitting a long sentence into two changes the delivery more reliably than any tag."}],[{"text":"Spell things out in the input.","b":true},{"text":" Writing “nineteen ninety-six” is often quicker than a tag and always works."}],[{"text":"Listen to the whole thing.","b":true},{"text":" Errors are local and a spot check will miss them."}],[{"text":"Chunk long reads at paragraph boundaries","b":true},{"text":", and check the joins for a level or timbre shift."}]]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"ul","items":[[{"text":"Piper","b":true},{"text":": fast, local, neural, dozens of languages, runs on a Raspberry Pi. The sensible default for free self-hosted TTS."}],[{"text":"eSpeak NG","b":true},{"text":": formant synthesis, very robotic, tiny, over a hundred languages, and still valuable for languages no neural model covers."}],[{"text":"Coqui TTS","b":true},{"text":": a widely used open-source toolkit; note that the company behind it shut down, so the repository is no longer actively maintained even though the models and code remain available and usable. Check the project’s state before building on it."}],[{"text":"Mimic 3","b":true},{"text":", "},{"text":"MaryTTS","b":true},{"text":": other open options."}],[{"text":"Whisper","b":true},{"text":" is the counterpart on the recognition side, covered in a later lesson."}]]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"Synthesised speech is now good enough that the remaining errors are more dangerous than the old obvious ones. A robotic voice signalled clearly that this was a machine; a natural voice confidently mispronouncing a drug name or a person’s surname does not signal anything, and a listener may not notice."}]},{"t":"p","runs":[{"text":"There is also no reliable way for the model to know what it got wrong. It does not have a confidence score for prosody, and it will read a sentence with exactly the wrong emphasis in a perfectly convincing voice. The only check is a person listening to the output against the intended meaning, which is unglamorous and is the actual quality control step."}]}],"check":{"question":[{"text":"A synthesised narration reads a list of seven product features in a uniform, slightly tiring way, although each sentence on its own sounds natural. What is the structural cause?"}],"options":[{"text":[{"text":"The vocoder is producing identical spectral characteristics for each sentence, because it has no conditioning on position within a document."}],"correct":false,"tempting":[{"text":"The vocoder converts an acoustic representation to a waveform; the flatness originates in the stage that produced that representation."}]},{"text":[{"text":"Most systems synthesise a sentence or chunk at a time with no memory of the paragraph, so emphasis cannot build and a list does not fall at the end."}],"correct":true,"tempting":null},{"text":[{"text":"The sentences are too similar in length and structure, and varying the sentence length would give the model enough signal to produce contour variation."}],"correct":false,"tempting":[{"text":"Varying the writing helps in practice, but the absence of cross-sentence context is the cause rather than the uniformity of the input."}]},{"text":[{"text":"The model has not been given an SSML prosody tag, and without an explicit rate or pitch instruction every synthesiser defaults to a fixed monotone contour derived from its training corpus’s mean fundamental frequency."}],"correct":false,"tempting":[{"text":"Systems produce varied contours within a sentence without any tags; the problem is scope, not a default monotone."}]}],"order":[2,3,0,1],"why":null},"prev":{"slug":"the-order-of-operations","title":"The chain, and why the order is not arbitrary","minutes":10},"next":{"slug":"how-a-voice-is-cloned","title":"The vector that is supposed to be you","minutes":11}}}