Robust universal neural vocoding

By Jaime Lorenzo-Trueba, Thomas Drugman, Javier Latorre, Thomas Merritt, Bartosz Putrycz, Roberto Barra-Chicote. 2018

19 citations recorded by the source corpus.

This paper introduces a robust universal neural vocoder trained with 74 speakers (comprised of both genders) coming from 17 languages. This vocoder is shown to be capable of generating speech of consistently good quality (98% relative mean MUSHRA when compared to natural speech) regardless of whether the input spectrogram comes from a speaker, style or recording condition seen during training or from an out-of-domain scenario. Together with the system, we present a full text-to-speech analysis of robustness of a number of implemented systems. The complexity of systems tested range from a convolutional neural networks-based system conditioned on linguistics to a recurrent neural networks-based system conditioned on mel-spectrograms. The analysis shows that convolutional neural networks-based systems are prone to occasional instabilities, while the recurrent approaches are significantly more stable and capable of providing universalizing robustness.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

In Other News: a Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Data (2019, methodology): “…as acoustic models which take text, in the form of characters or phonemes as input, and output low-level acoustic features that can be then converted into speech waveform using one of the several ‘vocoding’ techniques (Perraudin et al., 2013; Shen et al., 2018; Lorenzo-Trueba et al., 2018).”
A Neural Vocoder With Hierarchical Generation of Amplitude and Phase Spectra for Statistical Parametric Speech Synthesis (2019, methodology): “Neural vocoders based on these waveform generation models [16]–[21] have been developed to reconstruct speech wave-forms from various acoustic features for SPSS and some other tasks, such as voice conversion [22], [23], bandwidth extension [24] and speech coding [25].”
Attention Forcing for Sequence-to-sequence Model Training (2019, background): “φ can be considered a neural vocoder, which is not limited by the assumptions made by the conventional vocoders (Lorenzo-Trueba et al., 2018; Kalchbrenner et al., 2018).”
BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization (2020, methodology): “To complete the TTS pipeline, we convert mel-spectograms to waveforms using the multi-speaker neural vocoder of [19].”