The best Thai speech-to-text models drop every English word
Nobody had measured how good Thai speech-to-text actually is across the tools you'd reach for — the commercial APIs against the open models, on the same audio, scored the same way. So we did it: five systems, six clips of Thai from FLEURS, one scorer.
Here's what came back.
Why character error rate
Thai is written without spaces between words. Word error rate needs a tokenizer just to decide where one word ends, and every paper picks a different one, so the numbers don't compare across studies. Character error rate needs no tokenizer — you compare the character streams after one fixed normalization pass. Lower is better, and nobody can shave points by choosing a lenient tokenizer.
We track a second number too: how many of the English words in the reference survive into the transcript. Thai speakers drop English mid-sentence — "แล้วก็ meeting ตอน 3 โมง" — and a single CER hides whether the model caught "meeting" or replaced it with something Thai-shaped.
The board
| Model | Type | CER ↓ | Keeps English ↑ | Speed |
|---|---|---|---|---|
| Thonburian large-v3 | open | 6.9% | 0% | 6.0× |
| Whisper large-v3 | open | 9.0% | 50% | 6.0× |
| Typhoon ASR | open | 9.3% | 0% | 1.2× |
Deepgram Nova-3 (th) | commercial | 9.7% | 0% | 3.8× |
Deepgram Nova-3 (multi) | commercial | 99.3% | 25% | 14.4× |
Thonburian — a Thai fine-tune of Whisper — has the lowest character error. But it, Typhoon, and Deepgram's Thai model all score 0% English retention: they transcribe the Thai and drop every English word. Only plain Whisper large-v3 keeps the English (50%), and it pays about two points of Thai CER for it.
That's the finding: on this audio, being best at Thai and keeping the English are two different models, and none does both.
The one that fails outright
Deepgram Nova-3 has a multilingual auto-detect mode. It's the default a lot of integrations use — including, until we checked, ours. Thai isn't in that mode. Point it at Thai audio and it scores 99.3% CER: it romanizes the sound into something that isn't Thai at all. Switch it to the monolingual Thai model (language=th) and it's fine — 9.7%. If you're running Deepgram on Thai, check which mode you're in.
The caveats
Six clips of clean read speech. This is FLEURS — narrated, not conversational. The exact decimals will move as we add data. The shape — Thai-specialists win characters, lose English — is what's stable. The hard case isn't here yet. Real conversational Thai with English mixed in is where the English-retention gap should blow open, and it's exactly where there's no public, licensed benchmark to test on. Building that set is the main thing still owed. Every number reruns from the repo. No hand-edited cells, no rows we can't repeat. Onemodal run pulls the open models onto a GPU and scores them; the commercial APIs run behind their own keys.
The number that matters is yours
Six FLEURS clips show you the shape. Your actual audio — your speakers, your code-switching, your recording conditions — will land somewhere else. The reason the harness is reproducible is so you can point it at your own Thai and get your own numbers instead of trusting ours.
If your product speaks Thai and drops into English the way people actually talk, the model you want probably isn't the one at the top of this table — and the only way to know is to measure it on your data.
Interactive board and the full method: daslab.run/benchmark/thai-asr. Code and reproducible benchmarks: github.com/daslabhq/thai-asr-bench.
Why this, and why next
Thai is the first language we benchmarked because it's one we care about and one the general tools quietly get wrong. The method isn't Thai-specific: a fixed metric, the same clips through every system, commercial and open side by side, every number reproducible. German, Chinese, and the code-switch case that actually matters are next — and we'll publish those numbers too, whatever they say.