Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions

By Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Z. Chen, Yu Zhang, Yuxuan Wang, R. Skerry-Ryan and 3 more. 2017

3,120 citations recorded by the source corpus.

This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize time-domain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of 4.53 comparable to a MOS of 4.58 for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the conditioning input to WaveNet instead of linguistic, duration, and $F_{0}$ features. We further show that using this compact acoustic intermediate representation allows for a significant reduction in the size of the WaveNet architecture.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Tao Qin
  2. Tie-Yan Liu
  3. Xu Tan
  4. Yoshua Bengio
  5. Dong Yu
  6. Jinyu Li
  7. Mohit Bansal
  8. Quoc V. Le

In Natural Language Processing

  1. Graham Neubig
  2. Mohit Bansal
  3. Kyunghyun Cho
  4. Jianfeng Gao
  5. Tao Qin
  6. Tie-Yan Liu
  7. Quoc V. Le
  8. Luke S. Zettlemoyer

In Speech & Audio

  1. Jinyu Li
  2. Wei-Ning Hsu
  3. Xu Tan
  4. Yu Zhang
  5. Shinji Watanabe
  6. Junichi Yamagishi
  7. Hung-yi Lee
  8. Tao Qin

Citations in the corpus