EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture

By Chenfeng Miao, Shuang Liang, Zhencheng Liu, Minchuan Chen, Jun Ma, Shaojun Wang, Jing Xiao. 2020

46 citations recorded by the source corpus.

In this work, we address the Text-to-Speech (TTS) task by proposing a non-autoregressive architecture called EfficientTTS. Unlike the dominant non-autoregressive TTS models, which are trained with the need of external aligners, EfficientTTS optimizes all its parameters with a stable, end-to-end training procedure, while allowing for synthesizing high quality speech in a fast and efficient manner. EfficientTTS is motivated by a new monotonic alignment modeling approach (also introduced in this work), which specifies monotonic constraints to the sequence alignment with almost no increase of computation. By combining EfficientTTS with different feed-forward network structures, we develop a family of TTS models, including both text-to-melspectrogram and text-to-waveform networks. We experimentally show that the proposed models significantly outperform counterpart models such as Tacotron 2 and Glow-TTS in terms of speech quality, training efficiency and synthesis speed, while still producing the speeches of strong robustness and great diversity. In addition, we demonstrate that proposed approach can be easily extended to autoregressive models such as Tacotron 2.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Yu Zhang
6. Heiga Zen
7. Lei Xie
8. Dong Yu

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Yu Zhang
6. Heiga Zen
7. Lei Xie
8. Dong Yu

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “12 EfficientTTS [229] AM ch FF −→melS FF −→wav ICML21 2020.”
• Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis (2022, unknown): “Generally, with a well-trained neural acoustic model [2, 5, 6, 7] and a neural vocoder [8, 9, 10, 11], or alternatively using fully end-to-end models [12, 13, 14] which directly construct wave signals from text input, it is able to synthesize high-quality neutral speech.”
• DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs (2022, background): “They either leverage an external text-to-acoustic alignment module (Ren et al., 2019; 2021a; Peng et al., 2020; Elias et al., 2021) or jointly train one within the TTS model (Zeng et al., 2020; Miao et al., 2021; Badlani et al., 2021).”
• JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech (2022, methodology): “On the other hand, end-to-end text-to-speech (E2E-TTS) [5], [13], [14], [15], [16] is a recent research trend in which a speech waveform is directly generated from an input text in a single stage without distinction between an acoustic feature generator and a neural vocoder.”
• Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling (2021, methodology): “AlignTTS [18], JDI-T [21], and EfficientTTS [20] train an alignment network jointly with a TTS model to produce alignments between tokens and frames.”
• Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder For High Fidelity Flow-based Speech Synthesis (2021, background): “Integrating acoustic model and vocoder into an end-to-end model is another possible solution [4, 15, 16, 17], but this kind of models are notoriously difficult to train and extra Mel-spectrum based loss has to be adopted to stabilize the convergence process.”
• A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, unknown): “A line of fully end-to-end work adopts an adversarial decoder (or GAN), including FastSpeech 2 [87], EATS [15] and EFTS-Wav [65].”
• StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis (2022, unknown): “Although training internal aligners alleviates some generalization problems [1], [9], [38], [39], overfitting can occur as the aligners are trained on a smaller TTS dataset with only a mel-reconstruction loss.”