Universal Neural Vocoding with Parallel Wavenet

By Yunlong Jiao, Adam Gabrys, Georgi Tinchev, Bartosz Putrycz, Daniel Korzekwa, Viacheslav Klimkov. 2021

46 citations recorded by the source corpus.

We present a universal neural vocoder based on Parallel WaveNet, with an additional conditioning network called Audio Encoder. Our universal vocoder offers real-time high-quality speech synthesis on a wide range of use cases. We tested it on 43 internal speakers of diverse age and gender, speaking 20 languages in 17 unique styles, of which 7 voices and 5 styles were not exposed during training. We show that the proposed universal vocoder significantly outperforms speaker-dependent vocoders overall. We also show that the proposed vocoder outperforms several existing neural vocoder architectures in terms of naturalness and universality. These findings are consistent when we further test on more than 300 open-source voices.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Boris Ginsburg
7. Wei Ping
8. Yi Ren

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Wei Ping
6. Boris Ginsburg
7. Sheng Zhao
8. Sang-gil Lee

Citations in the corpus

• BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (2024, background): “During the meeting, where Coke executives debated the future of the company, Thomas, a young intern who had discovered a solution, mustered the courage to speak, shifting the direction of the conversation, that preceded his intervention.”
• A Survey on Neural Speech Synthesis (2021, background): “[141] further improve the robustness and universality of the vocoders.”
• BigVGAN: A Universal Neural Vocoder with Large-Scale Training (2022, methodology): “(2021) [14] builds the non-autoregressive universal vocoder with Parallel WaveNet [53].”
• Controllable Emphasis with zero data for text-to-speech (2023, methodology): “To produce speech, we vocode the mel-spectrograms frame using a universal vocoder [28].”
• CopyCat2: A Single Model for Multi-Speaker TTS and Many-to-Many Fine-Grained Prosody Transfer (2022, methodology): “All mel-spectrograms were synthesised using our universal neural vocoder [2].”
• eCat: An End-to-End Model for Multi-Speaker TTS & Many-to-Many Fine-Grained Prosody Transfer (2023, methodology): “Previously, Neural Text-to-Speech (NTTS) methods depended on training a separate component to generate mel-spectrograms from phonemized text [1–4] and another component (vocoder) to generate speech waveforms from mel-spectrograms [5–9].”
• Distribution Augmentation for Low-Resource Expressive Text-To-Speech (2022, unknown): “We use a multispeaker parallel WaveNet as a vocoder to produce the waveform for evaluations [23].”