FastSpeech 2: Fast and High-Quality End-to-End Text to Speech

By Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, Tie-Yan Liu. 2020

1,809 citations recorded by the source corpus.

Advanced text to speech (TTS) models such as FastSpeech can synthesize speech significantly faster than previous autoregressive models with comparable quality. The training of FastSpeech model relies on an autoregressive teacher model for duration prediction (to provide more information as input) and knowledge distillation (to simplify the data distribution in output), which can ease the one-to-many mapping problem (i.e., multiple speech variations correspond to the same text) in TTS. However, FastSpeech has several disadvantages: 1) the teacher-student distillation pipeline is complicated, 2) the duration extracted from the teacher model is not accurate enough, and the target mel-spectrograms distilled from teacher model suffer from information loss due to data simplification, both of which limit the voice quality. In this paper, we propose FastSpeech 2, which addresses the issues in FastSpeech and better solves the one-to-many mapping problem in TTS by 1) directly training the model with ground-truth target instead of the simplified output from teacher, and 2) introducing more variation information of speech (e.g., pitch, energy and more accurate duration) as conditional inputs. Specifically, we extract duration, pitch and energy from speech waveform and directly take them as conditional inputs during training and use predicted values during inference. We further design FastSpeech 2s, which is the first attempt to directly generate speech waveform from text in parallel, enjoying the benefit of full end-to-end training and even faster inference than FastSpeech. Experimental results show that 1) FastSpeech 2 and 2s outperform FastSpeech in voice quality with much simplified training pipeline and reduced training time; 2) FastSpeech 2 and 2s can match the voice quality of autoregressive models while enjoying much faster inference speed.

Subfields: Artificial Intelligence, Model Architectures, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Tao Qin
  2. Xu Tan
  3. Tie-Yan Liu
  4. Wei-Ning Hsu
  5. Dong Yu
  6. Yoshua Bengio
  7. Mohit Bansal
  8. James R. Glass

In Model Architectures

  1. Xu Tan
  2. Tao Qin
  3. Tie-Yan Liu
  4. Yi Ren
  5. Chenpeng Du
  6. Zhou Zhao
  7. Hung-yi Lee
  8. Jinglin Liu

In Speech & Audio

  1. Xu Tan
  2. Wei-Ning Hsu
  3. Jinyu Li
  4. Tao Qin
  5. Sheng Zhao
  6. Yossi Adi
  7. Lei He
  8. Hung-yi Lee

In Signal Processing

  1. Xu Tan
  2. Tao Qin
  3. Tie-Yan Liu
  4. Wei-Ning Hsu
  5. Jinyu Li
  6. Dong Yu
  7. Shinji Watanabe
  8. Hung-yi Lee

Citations in the corpus