Improve GAN-based Neural Vocoder using Truncated Pointwise Relativistic Least Square GAN
By Yanli Li, Congyi Wang. 2022
4 citations recorded by the source corpus.
Neural vocoders are widely utilized in modern text-to-speech (TTS) and voice conversion (VC) systems due to their high generation quality and fast inference speed. Recently, GAN-based neural vocoders have attracted great interest due to their lightweight and parallel structures, which enable them to generate a high-fidelity waveform in a real-time manner. Most existing GAN-based vocoders adopt the Least Square GAN (LSGAN) training framework. In this paper, we analyze the weaknesses of the LSGAN waveform synthesis framework and, inspired by Relativistic GAN, propose a simple yet effective variant of the LSGAN framework, named Truncated Pointwise Relativistic LSGAN (T-PRLSGAN). In this method, we consider the pointwise truism score distribution of real and fake wave segments and combine the Mean Squared error (MSE) loss with the proposed truncated pointwise relative discrepancy loss to increase the difficulty of the generator to fool the discriminator, leading to improved audio generation quality and stability. To demonstrate the effectiveness and generalization ability of our method, subjective and objective experiments have been conducted based on Avocodo, UnivNet, HiFiGAN, ParallelWaveGAN, and MelGAN vocoders, which show a consistent performance boost over those typical LSGAN-based vocoders. Moreover, our T-PRLSGAN can support multiple types of discriminators, i.e. multi-scale wave discriminator(MSD), multi-period discriminator(MPD), and multi-resolution spectrogram discriminator(MRD), without modifying their architecture or inference speed.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Cong Han
- Nima Mesgarani
- Gavin Mischler
- Vinay S. Raghavan
- Yi Luo
- John R. Hershey
- Jonathan Le Roux
- Yossi Adi
In Signal Processing
- Cong Han
- Gavin Mischler
- Vinay S. Raghavan
- Nima Mesgarani
- Yi Luo
- Jonathan Le Roux
- Yossi Adi
- Zhuo Chen
Citations in the corpus
- StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models (2023, background): “[47] along with the LSGAN loss functions [48] for decoder training, and incorporate the truncated pointwise relativistic loss function [49] to enhance sound quality (see Appendix F and G for details).”