VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention

By Peng Liu, Yuewen Cao, Songxiang Liu, Na Hu, Guangzhi Li, Chao Weng, Dan Su. 2021

24 citations recorded by the source corpus.

This paper proposes VARA-TTS, a non-autoregressive (non-AR) text-to-speech (TTS) model using a very deep Variational Autoencoder (VDVAE) with Residual Attention mechanism, which refines the textual-to-acoustic alignment layer-wisely. Hierarchical latent variables with different temporal resolutions from the VDVAE are used as queries for residual attention module. By leveraging the coarse global alignment from previous attention layer as an extra input, the following attention layer can produce a refined version of alignment. This amortizes the burden of learning the textual-to-acoustic alignment among multiple attention layers and outperforms the use of only a single attention layer in robustness. An utterance-level speaking speed factor is computed by a jointly-trained speaking speed predictor, which takes the mean-pooled latent variables of the coarsest layer as input, to determine number of acoustic frames at inference. Experimental results show that VARA-TTS achieves slightly inferior speech quality to an AR counterpart Tacotron 2 but an order-of-magnitude speed-up at inference; and outperforms an analogous non-AR model, BVAE-TTS, in terms of speech quality.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Xu Tan
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Dong Yu
7. Zhou Zhao
8. Yi Ren

In Signal Processing

1. Xu Tan
2. Tie-Yan Liu
3. Tao Qin
4. Frank K. Soong
5. Songxiang Liu
6. Dong Yu
7. Heeseung Kim
8. Sungroh Yoon

Citations in the corpus

• Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance (2021, methodology): “In addition, variational autoencoder based models (Kingma & Welling, 2014; Lee et al., 2020; Liu et al., 2021), diffusion based models (Ho et al.”
• A Survey on Neural Speech Synthesis (2021, methodology): “Replacing AR with NAR FastSpeech 1/2 [284, 286], ParaNet [262] EATS [67], Flow-TTS [228], [156, 200, 355, 73]”
• DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs (2022, methodology): “Other generative models have also been studied for TTS, such as Flow-based models(Kim et al., 2020; Miao et al., 2020), variational auto-encoder (VAE)-based models(Lee et al., 2021; Liu et al., 2021b), and generative adversarial network (GAN)-based models (Donahue et al., 2021; Yang et al., 2021).”