Parallel Neural Text-to-Speech

By Kainan Peng, Wei Ping, Zhao-long Song, Kexin Zhao. 2019

40 citations recorded by the source corpus.

In this work, we propose a non-autoregressive seq2seq model that converts text to spectrogram. It is fully convolutional and obtains about 46.7 times speed-up over Deep Voice 3 at synthesis while maintaining comparable speech quality using a WaveNet vocoder. Interestingly, it has even fewer attention errors than the autoregressive model on the challenging test sentences. Furthermore, we build the first fully parallel neural text-to-speech system by applying the inverse autoregressive flow~(IAF) as the parallel neural vocoder. Our system can synthesize speech from text through a single feed-forward pass. We also explore a novel approach to train the IAF from scratch as a generative model for raw waveform, which avoids the need for distillation from a separately trained WaveNet.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Yi Ren
5. Zhou Zhao
6. Sheng Zhao
7. Frank K. Soong
8. Wei-Ning Hsu

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Yi Ren
5. Sheng Zhao
6. Zhou Zhao
7. Wei-Ning Hsu
8. Boris Ginsburg

Citations in the corpus

• A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing (2022, neutral): “…will be extracted from one of this speaker’s other audio exam-2 ples. systems become capable of generating (Oord et al., 2016; Shen et al., et ; Peng et al., 2019; Ren et al., 2020). generally consist of two stages: stage which generates an intermediate acoustic representation (linear-or…”
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech (2020, background): “In recent years, non-autoregressive TTS models [11, 14, 16, 20] are designed to address these issues, which generate mel-spectrograms with extremely fast speed and avoid robustness issues, while achieving comparable voice quality with previous autoregressive models.”
• Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search (2020, methodology): “Another parallel TTS model, ParaNet (Peng et al., 2019), utilizes soft attention map from its teacher network.”
• End-to-End Adversarial Text-to-Speech (2020, neutral): “ParaNet [43] and FastSpeech [50] are non-autoregressive, but they require distillation [26] from an autoregressive model.”
• WaveFlow: A Compact Flow-based Model for Raw Audio (2019, background): “, 2019), ParaNet (Peng et al., 2019), and FastSpeech (Ren et al.”
• Semi-Supervised Speech Recognition via Local Prior Matching (2020, methodology): “Several methods can be used to estimate the text length on an unlabeled utterance, including the average speaking rate (Peng et al., 2019) or a phoneme/syllable segmentation (Adell & Bonafonte, 2004; Scharenborg et al., 2010; Wang et al., 2017).”
• Flow-TTS: A Non-Autoregressive Network for Text to Speech Based on Flow (2020, background): “No distillation Tacotron [5] ParaNet [15] Flow-TTS Tacotron 2 [6] FastSpeech [14] Deep Voice 3 [7] TransformerTTS [16] WaveNet [9] ParallelWaveNet [17] WaveGlow [10] WaveRNN [18] ClariNet [8] FloWaveNet [19] LPCNet [20] Table 1 .”
• Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, background): “Nonetheless, to achieve comparable speech quality with AR counterparts, NAR models usually require much more parameters [14], which may not be supported by edge a lightweight NAR model based adopts the same training scheme inaccurate duration extraction FastSpeech2 [13] alleviates [22] based…”
• Aligntts: Efficient Feed-Forward Text-to-Speech System Without Explicit Alignment (2020, background): “Most recently, many non-autoregressive architectures for TTS are proposed to increase the speed of synthesis speech, such as ParaNet [13] and FastSpeech [14].”
• EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture (2020, methodology): “In previous works, most non-autoregressive TTS models learn alignments from external models or tools (Peng et al., 2019; Ren et al., 2019; 2020), making the training complex.”
• Lightspeech: Lightweight and Fast Text to Speech with Neural Architecture Search (2021, methodology): “Non-autoregressiveTTS models [5, 9, 7, 8, 6] generate speech in parallel and greatly speed up the inference process compared to autoregressive TTS models [1, 2, 3].”
• TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model (2020, methodology): “As with other attention-based models, Tacotron, Transformer-TTS and ParaNet occasionally miss or repeat words [13].”
• Bunched LPCNet : Vocoder for Low-cost Neural Text-To-Speech Systems (2020, background): “The concept of generating multiple samples has been tried in the context of non-autoregressive vocoders [9, 10, 11, 12, 13] via parallel inference on GPUs.”
• Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework (2019, background): “…is more commonly sequential (Wang et al., 2017; Shen et al., 2018; Ping et al., 2017; Li et al., 2019) with notable exceptions (Ren et al., 2019; Peng et al., 2019), while the second stage, being much slower, is more commonly parallel (Oord et al., 2017; Prenger et al., 2018; Ping et al.,…”
• TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction (2021, methodology): “Attention-based models – Tacotron, Transformer-TTS and ParaNet – occasionally miss or repeat words [21].”
• Light-TTS: Lightweight Multi-Speaker Multi-Lingual Text-to-Speech (2021, background): “There are many studies that focus on increasing the speed of speech synthesis, such as DurIAN [5], ParaNet [6] and FastSpeech [7].”
• High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency (2020, unknown): “ParaNet [10], ClariNet [11] and GAN-TTS [12] are fully convolutional models that attempt to escape the autoregressiveness of the above models and synthesize speech in parallel.”