Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

By Isaac Elias, Heiga Zen, Jonathan Shen, Yu Zhang, Ye Jia, R. Skerry-Ryan, Yonghui Wu. 2021

85 citations recorded by the source corpus.

This paper introduces Parallel Tacotron 2, a non-autoregressive neural text-to-speech model with a fully differentiable duration model which does not require supervised duration signals. The duration model is based on a novel attention mechanism and an iterative reconstruction loss based on Soft Dynamic Time Warping, this model can learn token-frame alignments as well as token durations automatically. Experimental results show that Parallel Tacotron 2 outperforms baselines in subjective naturalness in several diverse multi speaker evaluations. Its duration control capability is also demonstrated.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models (2023, methodology): “Several recent works have advanced towards human-level TTS [3, 4, 5] using techniques like BERT pre-training [4, 40, 7] and E2E training [32, 5] with differentiable duration modeling [41, 42].”
A Survey on Neural Speech Synthesis (2021, methodology): “Replacing Attention with Duration FastSpeech [284], DurIAN [411], EATS [67] Glow-TTS [156], [190, 19, 355, 422, 73, 298]”
NaturalSpeech: End-to-End Text-to-Speech Synthesis With Human-Level Quality (2022, methodology): “as Tacotron 1/2 [4, 3], FastSpeech 1/2 [10, 18], ParallelTacotron 2 [42], Glow-TTS [13], and GradTTS [14], we are fully end-to-end with no cascaded errors.”
InstructTTS: Modelling Expressive TTS in Discrete Latent Space With Natural Language Style Prompt (2023, neutral): “Nowadays,themajorityofTTSsystems focusonusinganacousticmodel(AM)todirectlypredict mel-spectrogram,thenusesapre-trainedvocodertorecover waveformfromthepredictedmel-spectrogram[2],[25],[26].”
Translatotron 2: Robust direct speech-to-speech translation (2021, methodology): “C N ON - AUTOREGRESSIVE SYNTHESIZER Following recent non-autoregressive TTS works (Ren et al., 2021; Guo et al., 2021; Lee et al., 2021; Elias et al., 2021a), we explored using non-autoregressive synthesizer in Translatotron 2, which may have signiﬁcantly lower latency at inference time.”
Musika! Fast Infinite Waveform Music Generation (2022, unknown): “For example, several works focus on the task of inverting a low-dimensional audio representation (often a mel-spectrogram) back to the original waveform, which constitutes a building block of modern text-to-speech (TTS) systems [9–11].”
Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech (2022, background): “In [5, 7, 8], the information from the ground truth speech data, e.”
StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis (2022, unknown): “Although training internal aligners alleviates some generalization problems [1], [9], [38], [39], overﬁtting can occur as the aligners are trained on a smaller TTS dataset with only a mel-reconstruction loss.”
Phone-Level Prosody Modelling With GMM-Based MDN for Diverse and Controllable Speech Synthesis (2021, methodology): “Besides, non-autoregressive TTS models are also investigated for stability and fast generation, including FastSpeech2 [5] and parallel Tacotron2 [6].”
Translatotron 2: High-quality direct speech-to-speech translation with voice preservation (2021, methodology): “It is tempting to use a non-autoregressive architecture for the synthesizer of Translatotron 2, which may signiﬁcantly reduce its inference latency, similar to recent works on non-autoregressive TTS (Ren et al., 2019; 2021; Guo et al., 2021; Lee et al., 2021b; Elias et al., 2021b;a).”