Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

By Wei Ping, Kainan Peng, Andrew Gibiansky, Sercan Ö. Arik, Ajay Kannan, Sharan Narang, Jonathan Raiman, John Miller. 2017

494 citations recorded by the source corpus.

Subfields: Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Junichi Yamagishi
2. Tao Qin
3. Tie-Yan Liu
4. Xu Tan
5. Lei He
6. Yu Zhang
7. Mohit Bansal
8. Jianfeng Gao

In Model Architectures

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Hung-yi Lee
5. Yi Ren
6. Lei He
7. Junichi Yamagishi
8. Shinji Watanabe

Citations in the corpus

• DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, background): “, 2017b), the mel spectrogram from a text-to-spectrogram model (Ping et al., 2018; Shen et al., 2018), or the hidden states within the text-to-wave architecture (Ping et al.”
• Semi-Supervised Neural Architecture Search (2020, methodology): “Evaluation metric It has been shown that the quality of the attention alignment between the encoder and decoder is an important influence factor on the quality of synthesized speech in previous works [21, 31, 25, 13, 19], and misalignment can be observed for most mistakes (e.”
• High-Fidelity Audio Compression with Improved RVQGAN (2023, methodology): “To mitigate this problem, audio generation is typically divided into two stages: 1) predicting audio conditioned on some intermediate representation such as mel-spectrograms [23, 27, 19, 29] and 2) predicting the intermediate representation given some conditioning information, such as text [34, 33].”
• Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance (2021, methodology): “The autoregressive model is used for the text-to-intermediate feature model (Wang et al., 2017; Shen et al., 2018; Ping et al., 2018; Li et al., 2019) and vocoder (van den Oord et al., 2016; Kalchbrenner et al., 2018) to perform high-quality TTS.”
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis (2020, methodology): “…synthesis models use a two-stage pipeline: 1) predicting a low resolution intermediate representation such as mel-spectrograms (Shen et al., 2018, Ping et al., 2017, Li et al., 2019) or linguistic features (Oord et al., 2016) from text, and 2) synthesizing raw waveform audio from the…”
• Neural Voice Cloning with a Few Samples (2018, methodology): “Speaker embeddings have been shown to effectively capture speaker characteristics with low-dimensional vectors [Arik et al., 2017b, Ping et al., 2018].”
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis (2018, methodology): “For a baseline, we train a multi-speaker Tacotron similar to (Ping et al., 2017), using video IDs as a proxy for speaker labels.”
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech (2018, methodology): “, 2017b), Deep Voice 3 (Ping et al., 2018), Tacotron (Wang et al.”
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis (2019, methodology): “It has been successfully used to carry out text-to-speech synthesis, for example in Char2Wav, where WORLD vocoder features are modelled with an attention-based recurrent neural network (Sotelo et al., 2017; Shen et al., 2018; Ping et al., 2017).”
• Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search (2020, methodology): “One subclass of TTS models, including Tacotron 2 (Shen et al., 2018), Deep Voice 3 (Ping et al., 2017) and Transformer TTS (Li et al., 2019), generates a mel-spectrogram, a compressed representation of audio, from text.”
• FloWaveNet : A Generative Flow for Raw Audio (2018, neutral): “Copyright 2019 by the author(s). et al., 2018; Arik et al., 2017b;a; Ping et al., 2017; Jia et al., 2018).”
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training (2022, background): “…generation for out-of-distribution samples, is very valuable in many real-world applications, including text-to-speech with numerous speakers (Ping et al., 2018), neural voice cloning (Arik et al., 2018; Jia et al., 2018), voice conversion (Liu et al., 2018), speech-to-speech translation…”
• WaveFlow: A Compact Flow-based Model for Raw Audio (2019, methodology): “…waveforms, which can be conditioned on linguistic features (van den Oord et al., 2016; Arık et al., 2017a), the mel spectrograms from a text-tospectrogram model (Ping et al., 2018; Shen et al., 2018), or the learned hidden representation within a text-to-wave architecture (Ping et al., 2019).”
• Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis (2020, methodology): “…of the data while not requiring any additional Prenet or Postnet layer (Wang et al., 2017), nor compound loss functions required by most state of the art models like (Shen et al., 2017; Arik et al., 2017b;a; Ping et al., 2017; Skerry-Ryan et al., 2018; Wang et al., 2018; Bi´nkowski et al., 2019).”
• Fastpitch: Parallel Text-to-Speech with Pitch Prediction (2020, methodology): “The predominant paradigm in text-to-speech is two-stage synthesis: first producing mel-scale spec-trograms from text, and then the actual sound waves with a vocoder model [18, 23, 14].”
• MultiSpeech: Multi-Speaker Text to Speech with Transformer (2020, methodology): “Monotonic and diagonal alignments in the attention weights between text and speech are critical to ensure the quality of synthesized speech [12, 15, 16, 7, 17, 18].”
• Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, unknown): “Advances in deep learning have enabled TTS to achieve remarkable progress via sequence-to-sequence (seq2seq) acoustic models [7, 9-11], where the attention mechanism is used to align the text and speech automatically to achieve satisfactory synthesis results.”