WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN

By Pritish Chandna, Merlijn Blaauw, Jordi Bonada, Emilia G'omez. 2019

68 citations recorded by the source corpus.

We present a deep neural network based singing voice synthesizer, inspired by the Deep Convolutions Generative Adversarial Networks (DCGAN) architecture and optimized using the Wasserstein-GAN algorithm. We use vocoder parameters for acoustic modelling, to separate the influence of pitch and timbre. This facilitates the modelling of the large variability of pitch in the singing voice. Our network takes a block of consecutive frame-wise linguistic and fundamental frequency features, along with global singer identity as input and outputs vocoder features, corresponding to the block of features. This block-wise approach, along with the training methodology allows us to model temporal dependencies within the features of the input block. For inference, sequential blocks are concatenated using an overlap-add procedure. We show that the performance of our model is competitive with regards to the state-of-the-art and the original sample using objective metrics and a subjective listening test. We also present examples of the synthesis on a supplementary website and the source code via GitHub.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Xu Tan
3. Tie-Yan Liu
4. Zhou Zhao
5. Yi Ren
6. Jinglin Liu
7. Rongjie Huang
8. Yossi Adi

In Signal Processing

1. Zhou Zhao
2. Yi Ren
3. Jinglin Liu
4. Xu Tan
5. Rongjie Huang
6. Tao Qin
7. Tie-Yan Liu
8. Yossi Adi

Citations in the corpus

• DDSP: Differentiable Digital Signal Processing (2020, methodology): “Neural networks have been used previously to some success in modeling pre-extracted synthesis parameters (Blaauw & Bonada, 2017; Chandna et al., 2019), but these models fall short of end-to-end learning.”
• HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis (2020, methodology): “We mainly compare HiFiSinger with the following settings and systems: 1) Recording, the original singing recordings; 2) Recording (24kHz), the original singing recordings downsampled to 24kHz; 3) Xiaoic-eSing (Lu et al., 2020), a previous SVS system that also adopts 48kHz sampling rate but leverages”
• M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus (2022, background): “Recently, deep learning-based singing voice synthesis (SVS) has attracted a lot of attention from both industry and academic communities [4; 29; 21; 7; 5; 11; 9; 57; 26; 54; 41; 50].”
• Unsupervised Cross-Domain Singing Voice Conversion (2020, methodology): “Generative adversarial networks [23], conditioned on pitch and time-aligned phonemes, predicted a singing spectro-gram in a single pass instead of a frame-by-frame prediction.”