Char2Wav: End-to-End Speech Synthesis

By Jose M. R. Sotelo, Soroush Mehri, Kundan Kumar, João Felipe Santos, Kyle Kastner, Aaron C. Courville, Yoshua Bengio. 2017

460 citations recorded by the source corpus.

Subfields: Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Tie-Yan Liu
2. Tao Qin
3. Lei He
4. Junichi Yamagishi
5. Xu Tan
6. Yu Zhang
7. Quoc V. Le
8. Yoshua Bengio

In Speech & Audio

1. Xu Tan
2. Lei He
3. Junichi Yamagishi
4. Tao Qin
5. Yu Zhang
6. Hung-yi Lee
7. Tie-Yan Liu
8. Sheng Zhao

Citations in the corpus

• DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, neutral): “, 2018), Char2Wav (Sotelo et al., 2017), VoiceLoop (Taigman et al.”
• WaveGrad: Estimating Gradients for Waveform Generation (2020, background): “Deep generative models have revolutionized speech synthesis (Oord et al., 2016; Sotelo et al., 2017; Wang et al., 2017; Biadsy et al., 2019; Jia et al., 2019).”
• Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions (2017, methodology): “Char2Wav [16] describes yet another similar approach to end-to-end TTS using a neural vocoder.”
• Tacotron: Towards End-to-End Speech Synthesis (2017, background): “Char2Wav (Sotelo et al., 2017) is an independently-developed end-to-end model that can be trained on characters.”
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (2018, background): “There has been significant interest in end-to-end training of TTS models, which are trained directly from text-audio pairs, without depending on hand crafted intermediate representations [17, 23].”
• Neural Speech Synthesis with Transformer Network (2018, methodology): “Char2Wav (Sotelo et al. 2017) integrates the front-end and the back-end as one seq2seq (Sutskever, Vinyals, and Le 2014; Bahdanau, Cho, and Bengio 2014) model and learns the whole process in an end-to-end way, predicting acoustic parameters followed by a SampleRNN (Mehri et al. 2016) as the vocoder.”
• Deep Voice: Real-time Neural Text-to-Speech (2017, background): “Most recently, there has been a lot of work in parametric audio synthesis, notably WaveNet, SampleRNN, and Char2Wav (van den Oord et al., 2016; Mehri et al., 2016; Sotelo et al., 2017).”
• NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (2024, methodology): “Speech Representations in TTS. Traditional works propose using prior-based speech representation such as raw waveform [26, 27, 28] or mel-spectrogram [29, 30, 3, 31].”
• A Survey on Neural Speech Synthesis (2021, methodology): “Model One-Stage Training AR/NAR Modeling Architecture Char2Wav [309] N AR Seq2Seq RNN ClariNet [263] N AR Flow CNN FastSpeech 2s [286] Y NAR GAN Self-Att/CNN EATS [67] Y NAR GAN CNN Wave-Tacotron [379] Y AR Flow CNN/RNN/Hybrid EfficientTTS-Wav [229] Y NAR GAN CNN VITS [157] Y NAR VAE+Flow CNN/Self-A”
• LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech (2019, background): “The introduction of deep learning-based, neural end-to-end approaches has lowered the barrier to develop high-quality text-to-speech (TTS) systems [1–4].”
• Neural Voice Cloning with a Few Samples (2018, neutral): “, 2016], Char2Wav [Sotelo et al., 2017], Tacotron [Wang et al.”
• Fitting New Speakers Based on a Short Untranscribed Sample (2018, methodology): “The recent neural TTS systems include the Deep Voice systems DV1 (Arik et al., 2017b), DV2 (Arik et al., 2017a) and DV3 (Ping et al., 2018), WaveNet (Oord et al., 2016) and Parallel WaveNet (van den Oord et al., 2017), Char2Wav (Sotelo et al., 2017), Tacotron (Wang et al., 2017) and Tacotron2 (Shen ”
• Deep Voice 3: 2000-Speaker Neural Text-to-Speech (2017, background): “In contrast to Char2Wav and Tacotron, Deep Voice 3 avoids Recurrent Neural Networks (RNNs) 1 to speed up training and alleviates several challenging error modes that attention models fall into.”
• Deep Voice 2: Multi-Speaker Neural Text-to-Speech (2017, methodology): “Our contributions build upon recent work in entirely neural TTS systems, including Deep Voice 1 (Arik et al., 2017), Tacotron (Wang et al., 2017), and Char2Wav (Sotelo et al., 2017).”
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech (2018, methodology): “In contrast, Tacotron, Deep Voice 3, and Char2Wav employ the attention based sequence-to-sequence models (Bahdanau et al., 2015), yielding more compact architectures.”
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis (2019, methodology): “It has been successfully used to carry out text-to-speech synthesis, for example in Char2Wav, where WORLD vocoder features are modelled with an attention-based recurrent neural network (Sotelo et al., 2017; Shen et al., 2018; Ping et al., 2017).”
• LPCNET: Improving Neural Speech Synthesis through Linear Prediction (2018, background): “Neural speech synthesis algorithms have recently made it possible to both synthesize high-quality speech [1, 2, 3], and code high quality speech at very low bitrate [4].”
• Adversarial Audio Synthesis (2018, methodology): “, 2016) and Griffin-Lim (Griffin & Lim, 1984), or trained neural vocoders (Sotelo et al., 2017; Shen et al., 2018; Ping et al., 2018).”