Getting Malayalam TTS to Real-Time on CPU: What Worked, What Didn't
I fine-tuned F5-TTS on ~15 h of my own Youtube data (free Kaggle GPUs only) and benchmarked it on a 2-core CPU:
📉 64 steps → RTF 135 (CER 0.17)
⚡ 16 steps + EPSS → RTF 33.5, 4× faster (CER 0.22, same as 32 steps)
🎯 EPSS vs uniform steps at 7 steps: CER 0.26 vs 0.38
🔢 + INT8 → 1.27× more, no quality loss
🧩 ONNX Runtime → no gain; its INT8 version broke the output
Net: ~5× faster, still ~26× short of real time. Inference tricks have hit their limit; the rest must come from the model.
The goal: voice assistants for Malayalam, and other languages with little speech data, that run without GPUs, so communities with limited hardware can still build on speech AI. Speech recognition is next.
If you've distilled a flow-matching TTS model to a few steps, removed classifier-free guidance, or built a CPU-friendly single-speaker student model, I'd really value your pointers. Notebook available on request.
This is a genuine technical research update, not a promotional claim, and it holds up well — the two techniques it's built on (F5-TTS and EPSS) are both real, correctly named, and correctly described, and the reported numbers are internally consistent with what those techniques are known to do.
F5-TTS is real — a widely-used open-source flow-matching zero-shot TTS model (SWivid/F5-TTS), with an official fine-tuning workflow (f5-tts_finetune-gradio), so fine-tuning it on custom YouTube-derived audio is a standard, supported use case, not an unusual hack.
EPSS is a real, peer-reviewed technique, correctly cited and correctly applied. "Empirically Pruned Step Sampling" comes from Fast F5-TTS (Zheng et al., Interspeech 2025): by inspecting the sampling trajectory of F5-TTS, the authors identify redundant steps and propose EPSS, a non-uniform time-step sampling strategy that reduces the number of sampling steps while maintaining comparable quality — their own paper reports 7-step generation at 4x speedup on GPU with preserved performance, and explicitly notes EPSS generalizes to E2 TTS as well. Your finding that EPSS at 16 steps roughly matches the quality of double the uniform steps, and that EPSS clearly beats uniform sampling at very low step counts (7 steps), is exactly the behavior the original paper reports and the reason the technique exists.
The RTF/CER numbers are your own unverifiable benchmark results, but they're internally consistent and plausible, not implausible or too-good-to-be-true — a CPU-bound flow-matching diffusion transformer running dozens of denoising steps being 100+x slower than real-time on a 2-core CPU is exactly what you'd expect before any optimization, and the arithmetic across your own steps (RTF 135 → 33.5 → further ~1.27x from INT8) lines up to roughly the "~5x faster, ~26x short of real-time" conclusion you state.
The ONNX INT8 quality regression you hit is a known, commonly reported practical failure mode, not something suspicious — INT8 quantization via ONNX Runtime breaking output quality on transformer-based generative models (especially attention-heavy architectures) is a frequently documented issue in TTS/ASR deployment, so this isn't an unusual or implausible result.
There's nothing here to push back on — this reads as an honest, technically literate status update that undersells rather than oversells (explicitly stating the approach still falls well short of real-time), which is the opposite of the hype pattern in most of what's been checked in this thread.