GAN Vocoder: Multi-Resolution Discriminator Is All You Need

By Jaeseong You, Dalhyun Kim, Gyuhyeon Nam, Geumbyeol Hwang, Gyeongsu Chae. 2021

33 citations recorded by the source corpus.

Several of the latest GAN-based vocoders show remarkable achievements, outperforming autoregressive and flow-based competitors in both qualitative and quantitative measures while synthesizing orders of magnitude faster. In this work, we hypothesize that the common factor underlying their success is the multi-resolution discriminating framework, not the minute details in architecture, loss function, or training strategy. We experimentally test the hypothesis by evaluating six different generators paired with one shared multi-resolution discriminating framework. For all evaluative measures with respect to text-to-speech syntheses and for all perceptual metrics, their performances are not distinguishable from one another, which supports our hypothesis.

Subfields: Artificial Intelligence, Generative Models, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Xu Tan
3. Tie-Yan Liu
4. Zhou Zhao
5. Yi Ren
6. Yossi Adi
7. Jinglin Liu
8. Rongjie Huang

In Generative Models

1. Zhou Zhao
2. Yi Ren
3. Jinglin Liu
4. Rongjie Huang
5. Wei Ping
6. Xu Tan
7. Yossi Adi
8. Tao Qin

In Speech & Audio

1. Xu Tan
2. Tao Qin
3. Yossi Adi
4. Tie-Yan Liu
5. Zhou Zhao
6. Rongjie Huang
7. Yi Ren
8. Jinglin Liu

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Zhou Zhao
4. Rongjie Huang
5. Yi Ren
6. Jinglin Liu
7. Tie-Yan Liu
8. Yossi Adi

Citations in the corpus

• High Fidelity Neural Audio Compression (2022, background): “Multi scale discriminators are popular for capturing different structures in audio signals (Kumar et al., 2019; Kong et al., 2020; You et al., 2021).”
• Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis (2023, background): “It has been argued (You et al., 2021) that the architectural details of the generators do not significantly affect the vocoded outcome, given a well-established multi-resolution discriminating framework.”
• Audiodec: An Open-Source Streaming High-Fidelity Neural Audio Codec (2023, background): “Specifically, in contrast to the very lightweight generator, a generative adversarial network (GAN)-based model usually requires multiple deep discriminators to achieve highfidelity generation [33], and training these discriminators is timeconsuming.”
• A Survey on Neural Speech Synthesis (2021, background): “A lot of vocoders leverage GAN to ensure the audio generation quality, including WaveGAN [66], GAN-TTS [23], MelGAN [174], Parallel WaveGAN [396], HiFi-GAN [170], and other GAN-based vocoders [395, 385, 306, 410, 366, 134].”
• AudioGen: Textually Guided Audio Generation (2022, background): “Multi-scale discriminators are popular for capturing different structures in audio signals (Kumar et al., 2019; Kong et al., 2020; You et al., 2021).”
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training (2022, background): “Note that, You et al. (2021) argues that different generator architectures can perform equally well for single-speaker neural vocoding.”
• ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech (2022, methodology): “Previous neural TTS models [24, 42, 50] first generate mel-spectrograms autoregressively from text and then synthesize speech from the generated mel-spectrograms using a separately trained vocoder [10, 19, 53, 55].”
• HiFi++: a Unified Framework for Neural Vocoding, Bandwidth Extension and Speech Enhancement (2022, background): “Kong et al. (2020a); You et al. (2021) argue that the HiFi model success can be largely attributed to the multi-resolution discrimination framework.”