Tacotron: Towards End-to-End Speech Synthesis
By Yuxuan Wang, R. Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Z. Chen, Samy Bengio and 4 more. 2017
2,067 citations recorded by the source corpus.
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module. Building these components often requires extensive domain expertise and may contain brittle design choices. In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters. Given pairs, the model can be trained completely from scratch with random initialization. We present several key techniques to make the sequence-to-sequence framework perform well for this challenging task. Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness. In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Subfields: Artificial Intelligence, Model Architectures, Speech & Audio
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Model Architectures
In Speech & Audio
Citations in the corpus
- DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, neutral): “, 2018), Tacotron 1 & 2 (Wang et al., 2017; Shen et al., 2018), Char2Wav (Sotelo et al.”
- You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation (2020, positive): “Authors used Tacotron 2 as synthesizer, WaveNet as vocoder, and Wave2Letter+ ASR model from OpenSeq2Seq toolkit2.”
- Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learning (2019, methodology): “We followed the previous work of TTS [19] to extract spectrogram with the window size of 50 ms and the hop size of 12.5 ms.”
- An Unsupervised Autoregressive Model for Speech Representation Learning (2019, methodology): “The model is optimized by minimizing the L1 loss (as is done when predicting spectral frames in some speech synthesis models [25, 26].”
- MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, methodology): “We used a small number of mel spectrogram bins (64) because we found that at that threshold, we could reconstruct the original sequence at an acceptable level using the Griffin-Lim algorithm, [52] which itself might be a lower bound on quality as neural methods trained for this purpose have been sho”
- Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions (2017, methodology): “Tacotron [12], a sequence-to-sequence architecture [13] for producing magnitude spectrograms from a sequence of characters, simplifies the traditional speech synthesis pipeline by replacing the production of these linguistic and acoustic features with a single neural network trained from data alone.”
- Semi-Supervised Neural Architecture Search (2020, methodology): “Text to speech (TTS) [31, 25, 19, 13, 21] is an import task aiming to synthesize intelligible and natural speech from text.”
- Accelerating Protein Design Using Autoregressive Generative Models (2019, methodology): “Many applications in text-to-speech (Graves et al., 2013; Wang et al., 2017) and translation (Bahdanau et al., 2014; Sutskever et al., 2014) tackle context dependent word and audio prediction and generation using generative models with an autoregressive likelihood (Sutskever et al., 2011; LeCun et…”
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (2024, methodology): “Traditional TTS systems [1, 2, 3, 4] are typically trained on limited datasets recorded in studios, and thus fail to support high-quality zero-shot speech synthesis.”
- ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering (2024, methodology): “Later, end-to-end neural TTS models were introduced, which synthe-size Mel spectrograms and employ a vocoder (Oord et al., 2017; Prenger et al., 2019) for speech synthesis (Wang et al., 2017; Arık et al., 2017; Ren et al., 2019).”
- Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias (2023, methodology): “4) Content has monotonic alignment with speech while the autoregressive language model cannot guarantee that, which can lead to repeating or missing word issues [59, 58, 67].”
- PolyVoice: Language Models for Speech to Speech Translation (2023, methodology): “We evaluate our method on two speech-to-speech benchmark datasets, EMIME (Wester and Liang, 2011) and CVSS (Jia et al., 2022b).”
- VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation (2023, neutral): “For speech synthesis tasks, AED methods such as Tacotron (Wang et al., 2017), Tacotron2 (Shen et al., 2018), and Trans-formerTTS (Li et al., 2019), and duration-based methods, including Fastspeech (Ren et al., 2019), Fastspeech2 (Ren et al., 2020), and RobuTrans (Li et al., 2020) are widely used.”
- VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature (2022, methodology): “Compared with traditional statistical parametric speech synthesis [1], neural TTS model [2, 3] based on deep neural network shows a better performance.”