Flow-TTS: A Non-Autoregressive Network for Text to Speech Based on Flow

By Chenfeng Miao, Shuang Liang, Minchuan Chen, Jun Ma, Shaojun Wang, Jing Xiao. 2020

117 citations recorded by the source corpus.

In this work, we propose Flow-TTS, a non-autoregressive end-to-end neural TTS model based on generative flow. Unlike other non-autoregressive models, Flow-TTS can achieve high-quality speech generation by using a single feed-forward network. To our knowledge, Flow-TTS is the first TTS model utilizing flow in spectrogram generation network and the first non-autoregssive model which jointly learns the alignment and spectrogram generation through a single network. Experiments on LJSpeech show that the speech quality of Flow-TTS heavily approaches that of human and is even better than that of autoregressive model Tacotron 2 (outperforms Tacotron 2 with a gap of 0.09 in MOS). Meanwhile, the inference speed of Flow-TTS is about 23 times speed-up over Tacotron 2, which is comparable to FastSpeech.1

Subfields: Artificial Intelligence, Generative Models, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Zhou Zhao
5. Frank K. Soong
6. Sheng Zhao
7. Dong Yu
8. Yi Ren

In Generative Models

1. Xu Tan
2. Yi Ren
3. Zhou Zhao
4. Wei Ping
5. Tao Qin
6. Tie-Yan Liu
7. Frank K. Soong
8. Yu Zhang

In Speech & Audio

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Sheng Zhao
5. Yi Ren
6. Zhou Zhao
7. Frank K. Soong
8. Yu Zhang

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Yi Ren
5. Frank K. Soong
6. Zhou Zhao
7. Sheng Zhao
8. Yu Zhang

Citations in the corpus

• ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering (2024, methodology): “Some methods, utilizing techniques such as VAE (Hsu et al., 2019; Lee et al., 2022), flow (Miao et al., 2020; Kim et al., 2020), diffusion (Jeong et al., 2021; Kim et al., 2022; Popov et al., 2021), and others (Wu and Shi, 2022), have achieved promising performance in end-to-end speech synthesis.”
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias (2023, methodology): “Generative models, like language models [4, 33], VAE [34, 47], GAN [31, 30], Normalizing flow [39, 29], and diffusion model [32, 24, 43, 22], have been applied to speech or audio synthesis for years.”
• VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature (2022, methodology): “Hence, some research uses normalizing flow [21] in AM, such as FlowTTS [22] and GlowTTS [23].”
• A Survey on Neural Speech Synthesis (2021, methodology): “Some works such as ParaNet [262] and Flow-TTS [228] uses positional attention [264] for the text and speech alignment in parallel generation.”
• Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech (2021, background): “More recently, likelihood-based methods further eliminate the dependency on external aligners by estimating or learning alignments that maximize the likelihood of target mel-spectrograms (Zeng et al., 2020; Miao et al., 2020; Kim et al., 2020).”
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech (2020, background): “In recent years, non-autoregressive TTS models [11, 14, 16, 20] are designed to address these issues, which generate mel-spectrograms with extremely fast speed and avoid robustness issues, while achieving comparable voice quality with previous autoregressive models.”
• Diff-TTS: A Denoising Diffusion Model for Text-to-Speech (2021, methodology): “Additionally, the flow-based generative models such as Flow-TTS [18] and Glow-TTS [19] are parameter-inefficient due to architectural constraints imposed on normalizing flow-based models.”
• End-to-End Adversarial Text-to-Speech (2020, methodology): “Recent flow-based approaches Flow-TTS (Miao et al., 2020) and Glow-TTS (Kim et al.”
• Parallel Tacotron: Non-Autoregressive and Controllable TTS (2020, methodology): “Recently a number of neural TTS models relying on explicit phoneme duration rather than the soft attention mechanism were proposed [15–24].”
• ItôWave: Itô Stochastic Differential Equation is all You Need for Wave Generation (2022, background): “It was found that although the score network model does not have as strict restrictions on the network structure as the flow model [16, 7, 17], not all network structures are suitable for score prediction.”
• DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs (2022, methodology): “Other generative models have also been studied for TTS, such as Flow-based models(Kim et al., 2020; Miao et al., 2020), variational auto-encoder (VAE)-based models(Lee et al., 2021; Liu et al., 2021b), and generative adversarial network (GAN)-based models (Donahue et al., 2021; Yang et al., 2021).”
• PortaSpeech: Portable and High-Quality Generative Text-to-Speech (2021, methodology): “To generate high-quality mel-spectrograms, normalizing flows [8, 19] have been widely proved to be effective.”
• RAD-TTS: Parallel Flow-Based TTS with Robust Alignment Learning and Diverse Synthesis (2021, methodology): “Recent works (Kim et al., 2020; Miao et al., 2020; Peng et al., 2020) have presented increasingly parallel solutions to address these concern.”
• EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture (2020, methodology): “More recently, Flow-TTS (Miao et al., 2020), Glow-TTS (Kim et al.”
• VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention (2021, methodology): “, 2020) and Flow-TTS (Miao et al., 2020) are both flow-based non-AR TTS models.”