Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains

By Won Jang, D. Lim, Jaesam Yoon. 2020

42 citations recorded by the source corpus.

We propose Universal MelGAN, a vocoder that synthesizes high-fidelity speech in multiple domains. To preserve sound quality when the MelGAN-based structure is trained with a dataset of hundreds of speakers, we added multi-resolution spectrogram discriminators to sharpen the spectral resolution of the generated waveforms. This enables the model to generate realistic waveforms of multi-speakers, by alleviating the over-smoothing problem in the high frequency band of the large footprint model. Our structure generates signals close to ground-truth data without reducing the inference speed, by discriminating the waveform and spectrogram during training. The model achieved the best mean opinion score (MOS) in most scenarios using ground-truth mel-spectrogram as an input. Especially, it showed superior performance in unseen domains with regard of speaker, emotion, and language. Moreover, in a multi-speaker text-to-speech scenario using mel-spectrogram generated by a transformer model, it synthesized high-fidelity speech of 4.22 MOS. These results, achieved without external domain information, highlight the potential of the proposed model as a universal vocoder.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Lei Xie
7. Boris Ginsburg
8. Yi Ren

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Congyi Wang
6. Yanli Li
7. Wei Ping
8. Lei Xie

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “A lot of vocoders leverage GAN to ensure the audio generation quality, including WaveGAN [66], GAN-TTS [23], MelGAN [174], Parallel WaveGAN [396], HiFi-GAN [170], and other GAN-based vocoders [395, 385, 306, 410, 366, 134].”
• GAN Vocoder: Multi-Resolution Discriminator Is All You Need (2021, methodology): “However, recent GAN vocoders made breakthrough improvements [13, 14, 15, 16, 17], often exceeding the previous state-of-the-art WaveNet on various metrics both qualitatively (e.”
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training (2022, methodology): “Unlike previous studies which only adopted a subset ( train-clean-100 or train-clean-360 ) recorded in a clean environment (Jang et al., 2020; 2021; AlBadawy et al., 2022), we use all training data including the subset from diverse recording environments ( train-full = train-clean-100 +…”
• Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder For High Fidelity Flow-based Speech Synthesis (2021, methodology): “D denotes multi-resolution spectrum discriminator [11] operating on Mel-spectrum with different analysis parameters.”
• Improve GAN-based Neural Vocoder using Truncated Pointwise Relativistic Least Square GAN (2022, background): “UnivNet[5] and Universal MelGAN[4] further propose the multi-resolution spectrogram discriminators (MRD) which use a 2D convolution-based discrimi-nator in the waveform frequency domain, aiming at eliminating the artifacts directly in the frequency image, such as metallic and re-verb in hearing sens”