F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

By Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, Xie Chen. Annual Meeting of the Association for Computational Linguistics, 2024

490 citations recorded by the source corpus.

This paper introduces F5-TTS, a fully non-autoregressive text-to-speech system based on flow matching with Diffusion Transformer (DiT). Without requiring complex designs such as duration model, text encoder, and phoneme alignment, the text input is simply padded with filler tokens to the same length as input speech, and then the denoising is performed for speech generation, which was originally proved feasible by E2 TTS. However, the original design of E2 TTS makes it hard to follow due to its slow convergence and low robustness. To address these issues, we first model the input with ConvNeXt to refine the text representation, making it easy to align with the speech. We further propose an inference-time Sway Sampling strategy, which significantly improves our model's performance and efficiency. This sampling strategy for flow step can be easily applied to existing flow matching based models without retraining. Our design allows faster training and achieves an inference RTF of 0.15, which is greatly improved compared to state-of-the-art diffusion-based TTS models. Trained on a public 100K hours multilingual dataset, our F5-TTS exhibits highly natural and expressive zero-shot ability, seamless code-switching capability, and speed control efficiency. We have released all codes and checkpoints to promote community development, at https://SWivid.github.io/F5-TTS/.

Subfields: Signal Processing

View Paper: https://www.semanticscholar.org/paper/1434e344cb826e53a45bfed96a9d62f80fb4fcb2

Citations in the corpus

Qwen3-Omni Technical Report (2025, unknown): “Seed-TTS ICL (Anastassiou et al., 2024) 1.11 | 2.24 Seed-TTS RL (Anastassiou et al., 2024) 1.00 | 1.94 MaskGCT (Wang et al., 2024c) 2.27 | 2.62 E2 TTS (Eskimez et al., 2024) 1.97 | 2.19 F5-TTS (Chen et al., 2024c) 1.56 | 1.83 Spark TTS (Wang et al., 2025b) 1.20 | 1.98 CosyVoice 2 (Du et al., 2024) 1”
HierSpeech++: Bridging the Gap Between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-Shot Speech Synthesis (2023, unknown): “E2-TTS [44] introduced alignment-free TTS with ﬁller tokens, and F5-TTS [45] improved the robustness and e ﬃ ciency using DiT blocks and ConvNeXt-based text modeling.”
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models (2024, unknown): “Recent research indicates simpler approaches for text-speech alignment in NAR TTS models, such as E2 TTS [31], F5-TTS [32] and Seed-TTS [33].”