← All projects

Prototype Sep 2026

Uzbek TTS evaluation — 233 mispronunciations, one normaliser, a vendor memo

A commercial Uzbek text-to-speech engine misreads Roman numerals, decimals, initials and phone numbers. I measured how often, fixed it before synthesis, verified with a TTS→STT round trip, and sent the vendor the memo.

Role
Solo: evaluation design, corpus analysis, normaliser, memo
Stack
Python · regex · TTS/STT APIs · pytest
Roman-numeral cases in a 106k-char corpus
233
sites affected
37 / 43
tests in the normaliser
44
Before/after on the four confirmed cases, corpus counts, 44 tests
Before/after on the four confirmed cases, corpus counts, 44 tests

The problem

Synthesising narration for 43 heritage sites, the Uzbek voice read “XV asr” as letters, “2,5 km” as “two five”, and “M. Ulug’bek” as “em Ulugbek”. Historical text is full of these: one Roman numeral every ~455 characters.

What I did

  1. Measured instead of guessing: scanned the full corpus and counted the patterns (233 Roman numerals, 9 initials, 2 decimals, 0 phone numbers).
  2. Designed a 24-sentence probe across numbers, dates, money, units, Arabic names and four apostrophe encodings; listened, then confirmed objectively with a TTS→STT round trip (noting that STT adds its own errors — the ear is the final judge).
  3. Wrote a normaliser that touches only what the engine gets wrong: Roman numerals → ordinals with context (asr, bob, ranges like XIV–XV), decimals → “ikki yarim”, initials → full names from a dictionary or surname only, phones grouped with leading zeros. Years, money, percentages and units are deliberately left alone. 44 tests.
  4. Re-synthesised and verified: “o’n beshinchi asr”, “ikki yarim kilometr”, “Mirzo Ulug’bek va Alisher Navoiy” — correct.
  5. Sent the vendor a two-page memo with the probe set, scope numbers, the module and before/after audio.

Why it is here

It is the shape of work I want to do: a real production problem, a measurement, a fix with tests, a verification method with known limits, and a write-up someone else can act on.