Synthetic voice, and where it stops being good enough

What is solved and what is not

Modern text-to-speech produces speech that is clear, correctly pronounced, and in a convincing timbre. Cloning a recognisable voice from a short sample is a solved problem and it is cheap.

What is not solved is interpretation: deciding what a sentence means and delivering it accordingly. A synthetic voice reading a paragraph applies plausible-sounding prosody derived from the text’s surface. A human reading the same paragraph applies prosody derived from understanding it.

Over one sentence the difference is invisible. Over three minutes it accumulates into a specific fatigue: the listener stops believing anyone is there. That is the honest boundary of the technology and it is not about audio quality.

The quality ladder, with the free rungs named

From most accessible upward:

• Piper. Small, fast, runs on a Raspberry Pi or an old laptop, entirely offline. Robotic by current standards and perfectly usable for short utterances, accessibility and prototyping. Free and open.
• Kokoro. A recent small open model with a notably good quality-to-size ratio. Runs locally and in a browser Space. Free.
• Coqui XTTS. Multilingual, supports voice cloning from a short sample, runs locally on modest hardware. Free and open; the company behind it wound down, which is a genuine caution about long-term maintenance rather than about the model’s usefulness today.
• Hosted services, of which ElevenLabs is the best known. Clearly better on expressiveness and on long-form stability. Paid, with small free tiers.

For most of this readership, a free model plus careful text preparation gets within touching distance of a paid one on short lines. On a five-minute narration, the paid ones pull ahead noticeably.

Getting more out of any of them

The text is an instrument. Most people type a paragraph and press generate.

• Punctuate for breath, not for grammar. A comma inserts a short pause. A full stop inserts a longer one. Adding commas where you want the speaker to breathe works better than any setting.
• Break long sentences. Synthetic prosody degrades across long clauses. Two short sentences outperform one long one.
• Spell out anything ambiguous. Numbers, dates, acronyms, units. “2026” may be read as a number rather than a year; “twenty twenty-six” is not ambiguous. “Dr” is doctor or drive.
• Generate line by line, not paragraph by paragraph. You can then regenerate the one bad line rather than the whole take, and assemble in the editor with pauses of your own choosing.
• Use the emotion or style controls where the model has them, and test them rather than trusting their names.

Cloning, and the permission it requires

Cloning a voice needs seconds of audio. That is the technical fact and it is why the rule has to be firm rather than proportionate to effort.

A person’s voice is theirs. Using it requires their written, specific permission: what it will say, where it will appear, for how long, and whether new sentences may be generated. Recording somebody for one purpose does not license synthesising them for another. This is the same principle as the likeness rule and it is stated again because the ease of cloning makes it easy to forget that anything was taken.

Voice is also increasingly legislated for directly: several US states’ publicity-rights statutes name voice explicitly, and courts in a number of countries have treated a distinctive voice as protectable. Where you are and where your audience is both matter, and neither this lesson nor any lesson is legal advice.

For your own voice, cloning is straightforward and useful: record a clean sample once, then produce corrections and pickups without setting up a microphone again.

Languages and accents

A practical point for this audience. Multilingual models vary enormously in quality by language, and quality in a language’s majority accent does not predict quality in others. Hindi and Hinglish output from a model trained mostly on English can be intelligible and audibly wrong in rhythm.

Test the specific language and accent you need before building a project on it. If the result is not good, a real speaker recorded on a phone is better, cheaper and faster than fighting the model.

Cleaning up whatever you generate

Synthetic speech benefits from the same treatment as recorded speech: trim the silences, normalise the level, apply gentle compression, and cut any low-frequency rumble. Audacity does all of it for free on any machine, and it takes two minutes.

The next lesson covers that chain properly, because it applies identically to a real recording.

Today: generate one line three times: as a paragraph, as a single sentence, and with commas inserted where you want breaths. Listen to all three.