DiffVoice: Text-to-Speech with Latent Diffusion

By Zhijun Liu, Yiwei Guo, K. Yu. 2023

32 citations recorded by the source corpus.

In this work, we present DiffVoice, a novel text-to-speech model based on latent diffusion. We propose to first encode speech signals into a phoneme-rate latent representation with a variational autoencoder enhanced by adversarial training, and then jointly model the duration and the latent representation with a diffusion model. Subjective evaluations on LJSpeech and LibriTTS datasets demonstrate that our method beats the best publicly available systems in naturalness. By adopting recent generative inverse problem solving algorithms for diffusion models, DiffVoice achieves the state-of-the-art performance in text-based speech editing, and zero-shot adaptation.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yiwei Guo
2. Xie Chen
3. Chenpeng Du
4. K. Yu
5. Zhijun Liu
6. Feiyu Shen
7. Zheng Liang
8. Shuai Wang

In Signal Processing

1. Yiwei Guo
2. Xie Chen
3. Chenpeng Du
4. K. Yu
5. Zhijun Liu
6. Feiyu Shen
7. Dong Yu
8. Jinyu Li

Citations in the corpus

• UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding (2023, methodology): “Additional techniques, such as normalizing flow (Valle et al. 2020; Kim et al. 2020) and diffusion models (Liu et al. 2022; Popov et al. 2021; Liu, Guo, and Yu 2023), have been introduced to generate the mel-spectrogram.”
• StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis (2022, unknown): “A similar discrepancy has been reported in a very recent study that examines the effects of evaluation methods on the MOS results [62], and our evaluation of VITS is comparable to other studies that have tried to reproduce it on both LJSpeech and LibriTTS datasets [63], [64], [65], [66].”