SingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation

By Feiyang Chen, Rongjie Huang, Chenye Cui, Yi Ren, Jinglin Liu, Zhou Zhao, Nicholas Jing Yuan, Baoxing Huai. 2021

74 citations recorded by the source corpus.

Deep generative models have achieved significant progress in speech synthesis to date, while high-fidelity singing voice synthesis is still an open problem for its long continuous pronunciation, rich high-frequency parts, and strong expressiveness. Existing neural vocoders designed for text-to-speech cannot directly be applied to singing voice synthesis because they result in glitches and poor high-frequency reconstruction. In this work, we propose SingGAN, a generative adversarial network designed for high-fidelity singing voice synthesis. Specifically, 1) to alleviate the glitch problem in the generated samples, we propose source excitation with the adaptive feature learning filters to expand the receptive field patterns and stabilize long continuous signal generation; and 2) SingGAN introduces global and local discriminators at different scales to enrich low-frequency details and promote high-frequency reconstruction; and 3) To improve the training efficiency, SingGAN includes auxiliary spectrogram losses and sub-band feature matching penalty loss. To the best of our knowledge, SingGAN is the first work designed toward high-fidelity singing voice vocoding. Our evaluation of SingGAN demonstrates the state-of-the-art results with higher-quality (MOS 4.05) samples. Also, SingGAN enables a sample speed of 50x faster than real-time on a single NVIDIA 2080Ti GPU. We further show that SingGAN generalizes well to the mel-spectrogram inversion of unseen singers, and the end-to-end singing voice synthesis system SingGAN-SVS enjoys a two-stage pipeline to transform the music scores into expressive singing voices.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhou Zhao
2. Rongjie Huang
3. Yi Ren
4. Jinglin Liu
5. Dong Yu
6. Chenye Cui
7. Chao Weng
8. Dongchao Yang

In Signal Processing

1. Rongjie Huang
2. Zhou Zhao
3. Yi Ren
4. Jinglin Liu
5. Chenye Cui
6. Dongchao Yang
7. Chao Weng
8. Dong Yu

Citations in the corpus

• Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners (2024, positive): “For singing voice generation, we further include F0-driven source excitation to stabilize long-continuous waveforms generation following (Liu et al., 2022; Huang et al., 2022a).”
• ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech (2022, methodology): “Previous neural TTS models [24, 42, 50] first generate mel-spectrograms autoregressively from text and then synthesize speech from the generated mel-spectrograms using a separately trained vocoder [10, 19, 53, 55].”
• Make-A-Voice: Unified Voice Synthesis With Discrete Representation (2023, background): “The limited amount of labeled data [7] and unpleasant glitches [13] are two major challenges for singing”
• M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus (2022, unknown): “…continuous breakthroughs in TTS techniques [47; 23; 52; 37; 35; 36; 18], singing voice synthesis [45; 21; 14; 5; 38; 2; 27; 40; 9; 13; 10; 51; 17; 8; 43; 1; 22; 48; 54; 41; 19; 56; 53; 25] and its related tasks (e.g., SVC [33; 42; 24; 46; 55]) have received extensive attention in both industry…”