Adaptive Wavenet Vocoder for Residual Compensation in GAN-Based Voice Conversion
By Berrak Sisman, Mingyang Zhang, Sakriani Watiasri Sakti, Haizhou Li, Satoshi Nakamura. 2018
38 citations recorded by the source corpus.
In this paper, we propose to use generative adversarial networks (GAN) together with a WaveNet vocoder to address the over-smoothing problem arising from the deep learning approaches to voice conversion, and to improve the vocoding quality over the traditional vocoders. As GAN aims to minimize the divergence between the natural and converted speech parameters, it effectively alleviates the over-smoothing problem in the converted speech. On the other hand, WaveNet vocoder allows us to leverage from the human speech of a large speaker population, thus improving the naturalness of the synthetic voice. Furthermore, for the first time, we study how to use WaveNet vocoder for residual compensation to improve the voice conversion performance. The experiments show that the proposed voice conversion framework consistently outperforms the baselines.
Subfields: Artificial Intelligence, Generative Models, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Haizhou Li
- Berrak Sisman
- Junichi Yamagishi
- Xu Tan
- Sakriani Watiasri Sakti
- Satoshi Nakamura
- Andros Tjandra
- Rui Liu
In Generative Models
- Haizhou Li
- Berrak Sisman
- Junichi Yamagishi
- Simon King
- Kun Zhou
- Satoshi Nakamura
- Sakriani Watiasri Sakti
- Guanglai Gao
In Speech & Audio
- Junichi Yamagishi
- Sakriani Watiasri Sakti
- Satoshi Nakamura
- Andros Tjandra
- Haizhou Li
- Berrak Sisman
- Mingyang Zhang
- Xin Eric Wang
In Signal Processing
- Haizhou Li
- Berrak Sisman
- Junichi Yamagishi
- Sakriani Watiasri Sakti
- Satoshi Nakamura
- Andros Tjandra
- Rui Liu
- Simon King
Citations in the corpus
- VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019 (2019, methodology): “We also attempted further enhancement of the synthesized voice using several techniques, such as WaveNet [26, 27] and GAN-based voice conversion [28].”
- Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS (2020, background): “The recent progress in neural vocoder [4], [11]–[15] also contributes to the improvement of speech quality.”
- Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet (2019, methodology): “To make use of non-parallel source-target speech corpora, voice conversion based on the variational autoencoder (VAE) [13] and cycleconsistent generative adversarial network [14, 15, 16, 17] were proposed.”
- Expressive TTS Training With Frame and Style Reconstruction Loss (2020, background): “a neural vocoder [5], [16]–[21], neural TTS generates naturalsounding and human-like speech which achieves state-of-the-art”
- Emotional Voice Conversion: Theory, Databases and ESD (2021, background): “At run-time, the system converts the speaker identity from one to another [69, 18].”
- An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning (2020, methodology): “GANs offer a general framework for training a data generator in such a way that it can deceive a real/fake discriminator that attempts to distinguish real data and fake data produced by the generator [242]–[244].”
- Teacher-Student Training For Robust Tacotron-Based TTS (2019, background): “Furthermore, together with a neural vocoder [5, 11–15], natural-sounding human-like speech can be generated.”