DCTTS: Discrete Diffusion Model with Contrastive Learning for Text-to-Speech Generation

By Zhichao Wu, Qiulin Li, Sixing Liu, Qun Yang. 2023

12 citations recorded by the source corpus.

In the Text-to-speech(TTS) task, the latent diffusion model has excellent fidelity and generalization, but its expensive resource consumption and slow inference speed have always been a challenging. To address this issue, this paper proposes the Discrete Diffusion Model with Contrastive Learning for Text-to-Speech Generation(DCTTS). Specifically, we employs a straightforward and effective text encoder, compresses the raw data into discrete space using VQ model, and then trains the diffusion model on the discrete space. In order to minimize the number of diffusion steps needed to synthesis high-quality speech, we used a contrastive learning loss throughout the diffusion model training phase. The experimental results demonstrate that the approach proposed in this paper has outstanding speech synthesis quality and sampling speed while significantly reducing the resource consumption of diffusion model. The synthesized samples are available at https://github.com/lawtherWu/DCTTS

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Sheng Zhao
3. Yanqing Liu
4. Tao Qin
5. Tie-Yan Liu
6. Jinyu Li
7. Yichong Leng
8. Dongchao Yang

In Signal Processing

1. Xu Tan
2. Sheng Zhao
3. Yanqing Liu
4. Jinyu Li
5. Yichong Leng
6. Jiang Bian
7. Zeqian Ju
8. Kai Shen

Citations in the corpus

• NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (2024, methodology): “We use Librilight [56], which contains 60 K hours of 16 KHz unlabeled speech data and around 7000 distinct speakers from LibriVox audiobooks, as the training set.”