Towards Robust Neural Vocoding for Speech Generation: A Survey

By Po-Chun Hsu, Chunhui Wang, Andy T. Liu, Hung-yi Lee. 2019

26 citations recorded by the source corpus.

Recently, neural vocoders have been widely used in speech synthesis tasks, including text-to-speech and voice conversion. However, when encountering data distribution mismatch between training and inference, neural vocoders trained on real data often degrade in voice quality for unseen scenarios. In this paper, we train four common neural vocoders, including WaveNet, WaveRNN, FFTNet, Parallel WaveGAN alternately on five different datasets. To study the robustness of neural vocoders, we evaluate the models using acoustic features from seen/unseen speakers, seen/unseen languages, a text-to-speech model, and a voice conversion model. We found out that the speaker variety is much more important for achieving a universal vocoder than the language. Through our experiments, we show that WaveNet and WaveRNN are more suitable for text-to-speech models, while Parallel WaveGAN is more suitable for voice conversion applications. Great amount of subjective MOS results in naturalness for all vocoders are presented for future studies.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Lei He
6. Sheng Zhao
7. Junichi Yamagishi
8. Erica Cooper

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “[115] study the robustness of vocoders by evaluating several common vocoders with comprehensive experiments.”
• A critical evaluation, challenges, and future perspectives of using artificial intelligence and emerging technologies in smart classrooms (2023, background): “All three topics are based on AI approaches, like GAN deep neural networks, in order to enhance the quality of the generated speech (Hsu et al., 2019) and reasoning, that helps bots make predictions and draw conclusions with the aim to respond appropriately in every interaction with a human.”
• Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “The powerful VC techniques based on neural network [208], vocoder [209, 210], GAN [211-”
• Can Speaker Augmentation Improve Multi-Speaker End-to-End TTS? (2020, background): “[19, 20]), improvements are rather small and the best augmentation strategy for speech synthesis is still unknown.”
• An Efficient Subband Linear Prediction for LPCNet-Based Neural Synthesis (2020, background): “Moreover, it also shows the robustness in several scenarios[6, 7].”
• Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains (2020, unknown): “A vocoder that can generate high-fidelity audio in various domains, regardless of whether the input has been encountered during training or has come from an out-of-domain source, is usually called a universal vocoder[10, 11].”