Seed-TTS: A Family of High-Quality Versatile Speech Generation Models

By Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao and 36 more. arXiv.org, 2024

395 citations recorded by the source corpus.

We introduce Seed-TTS, a family of large-scale autoregressive text-to-speech (TTS) models capable of generating speech that is virtually indistinguishable from human speech. Seed-TTS serves as a foundation model for speech generation and excels in speech in-context learning, achieving performance in speaker similarity and naturalness that matches ground truth human speech in both objective and subjective evaluations. With fine-tuning, we achieve even higher subjective scores across these metrics. Seed-TTS offers superior controllability over various speech attributes such as emotion and is capable of generating highly expressive and diverse speech for speakers in the wild. Furthermore, we propose a self-distillation method for speech factorization, as well as a reinforcement learning approach to enhance model robustness, speaker similarity, and controllability. We additionally present a non-autoregressive (NAR) variant of the Seed-TTS model, named $\text{Seed-TTS}_\text{DiT}$, which utilizes a fully diffusion-based architecture. Unlike previous NAR-based TTS systems, $\text{Seed-TTS}_\text{DiT}$ does not depend on pre-estimated phoneme durations and performs speech generation through end-to-end processing. We demonstrate that this variant achieves comparable performance to the language model-based variant and showcase its effectiveness in speech editing. We encourage readers to listen to demos at \url{https://bytedancespeech.github.io/seedtts_tech_report}.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/61afadf4f179a82800f6e4b943709d75bbabb45c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ziyang Ma
2. Zhikang Niu
3. Shuai Bai
4. Kai Dang
5. Yunfei Chu
6. Jian Zhao
7. Xie Chen
8. Keqi Deng

In Signal Processing

1. Ziyang Ma
2. Zhikang Niu
3. Jian Zhao
4. Xie Chen
5. Keqi Deng
6. Yushen Chen
7. Chunhui Wang
8. Kai Yu

Citations in the corpus

• Qwen3-Omni Technical Report (2025, neutral): “Due to the lack of relevant assessments, the evaluation of speech generation focuses primarily speech generation given texts, similarity to text-to-speech (TTS), on following three aspects: • Zero-Shot Speech Generation : We assess the content consistency (WER) and speaker similarity (SIM) of our mo”
• F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching (2024, methodology): “…We compare our models with leading TTS systems including, (mainly) autoregressive models: VALL-E 2 [13], MELLE [29], FireRedTTS [63] and CosyVoice [64]; non-autoregressive models: Voicebox [14], NaturalSpeech 3 [12], DiTTo-TTS [35], MaskGCT [65], Seed-TTS DiT [39] and our reproduced E2 TTS [36].”
• Qwen2.5-Omni Technical Report (2025, positive): “5-Omni achieves 1.42%, 2.33% and 6.54% WER on seed-tts-eval (Anastassiou et al., 2024) test-zh, test-en and test-hard set respectively, outperforming MaskGCT (Wang et al., 2024e) and CosyVoice 2 (Du et al., 2024).”
• CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models (2024, neutral): “Recent research indicates simpler approaches for text-speech alignment in NAR TTS models, such as E2 TTS [31], F5-TTS [32] and Seed-TTS [33].”
• HierSpeech++: Bridging the Gap Between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-Shot Speech Synthesis (2023, neutral): “We also compared the model with the recently proposed AR, di ff usion, and CFM-based zero-shot TTS models: 1) MaskGCT [88], Seed-TTS [89], FireRedTTS [90], E2-TTS [44], F5-TTS [45], and CosyVoice2 [46].”