Expediting TTS Synthesis with Adversarial Vocoding

By Paarth Neekhara, Chris Donahue, Miller S. Puckette, Shlomo Dubnov, Julian McAuley. 2019

23 citations recorded by the source corpus.

Recent approaches in text-to-speech (TTS) synthesis employ neural network strategies to vocode perceptually-informed spectrogram representations directly into listenable waveforms. Such vocoding procedures create a computational bottleneck in modern TTS pipelines. We propose an alternative approach which utilizes generative adversarial networks (GANs) to learn mappings from perceptually-informed spectrograms to simple magnitude spectrograms which can be heuristically vocoded. Through a user study, we show that our approach significantly outperforms naive vocoding strategies while being hundreds of times faster than neural network vocoders used in state-of-the-art TTS systems. We also show that our method can be used to achieve state-of-the-art results in unsupervised synthesis of individual words of speech.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

A Survey on Neural Speech Synthesis (2021, neutral): “12 AdVoc [239] Voc melS FF −→linS→wav IS19 2019.”
MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis (2019, methodology): “Neekhara et al. (2019) propose using GANs to learn mappings from mel-spectrogram to simple magnitude spectrogram, which is to be combined with phase estimations to recover raw audio waveform.”
It's Raw! Audio Generation with State-Space Models (2022, methodology): “For evaluating mean opinion scores (MOS), we repurpose scripts for creating jobs for Amazon Mechanical Turk from Neekhara et al. [29].”
High Fidelity Speech Synthesis with Adversarial Networks (2019, neutral): “Neekhara et al. (2019) propose an adversarial vocoder model that is able to synthesise magnitude spectrograms from mel-spectrograms generated by Tacotron 2, and combine this with phase estimation using the Local Weighted Sums technique (Le Roux et al., 2010).”
End-to-End Adversarial Text-to-Speech (2020, neutral): “Neekhara et al. [41] predict magnitude spectrograms from mel-spectrograms.”
Expressive Neural Voice Cloning (2021, methodology): “(2) Vocoding the synthesized spectrograms to audible waveforms using a neural vocoder [27, 22, 18] or heuristic methods like the Griﬃn-Lim [8] algorithm.”
ISTFTNET: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform (2022, methodology): “The use of iSTFT for neural speech synthesis was previously introduced [26, 27, 28] (including our own early attempts [26]).”