Aligntts: Efficient Feed-Forward Text-to-Speech System Without Explicit Alignment

By Zhen Zeng, Jianzong Wang, Ning Cheng, Tian Xia, Jing Xiao. 2020

63 citations recorded by the source corpus.

Targeting at both high efficiency and performance, we propose AlignTTS to predict the mel-spectrum in parallel. AlignTTS is based on a Feed-Forward Transformer which generates mel-spectrum from a sequence of characters, and the duration of each character is determined by a duration predictor. Instead of adopting the attention mechanism in Transformer TTS to align text to mel-spectrum, the alignment loss is presented to consider all possible alignments in training by use of dynamic programming. Experiments on the LJSpeech dataset show that our model achieves not only state-of-the-art performance which outperforms Transformer TTS by 0.03 in mean option score (MOS), but also a high efficiency which is more than 50 times faster than real-time.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Yi Ren
7. Zhou Zhao
8. Yu Zhang

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Replacing Attention with Duration FastSpeech [284], DurIAN [411], EATS [67] Glow-TTS [156], [190, 19, 355, 422, 73, 298]”
• Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech (2021, background): “More recently, likelihood-based methods further eliminate the dependency on external aligners by estimating or learning alignments that maximize the likelihood of target mel-spectrograms (Zeng et al., 2020; Miao et al., 2020; Kim et al., 2020).”
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech (2020, methodology): “Compared with previous non-autoregressive acoustic models [7, 11, 28], most of them focus on improving the duration accuracy to reduce the information gap between the input and output, while FastSpeech 2 and 2s take more missing information into account.”
• Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search (2020, methodology): “In parallel with our work, AlignTTS [34], Flowtron [28], and Flow-TTS [15] have been proposed.”
• Parallel Tacotron: Non-Autoregressive and Controllable TTS (2020, background): “Some of these models train their explicit duration model to predict groundtruth phoneme duration extracted from autoregressive models [15, 18, 22], internal aligner models [17, 19], or external aligner models [16, 23, 24].”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “For example, AlignTTS [254] uses the dynamic programming to consider all possible alignments in training, that is, uses the alignment loss inspired by the Baum-Welch algorithm [11, 206] to train the mix density network for alignment.”
• Fast DCTTS: Efficient Deep Convolutional Text-to-Speech (2021, methodology): “Recent works including FastSpeech [6], AlignTTS [7], and FastSpeech2 [8] apply non-autoregressive architecture and significantly improve synthesis speed.”
• Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling (2021, methodology): “The proposed Parallel Tacotron 2 is a duration-based non-autoregressive neural TTS model which does not require supervised duration signals, like [18–21 AlignTTS [18], JDI-T [21], and EfficientTTS [20] train an alignment network jointly with a TTS model to produce alignments between tokens and frames.”
• LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation (2021, unknown): “To verify the effectiveness of the proposed model as the vocoder in the TTS framework, we combine it with Transformer TTS [18] and AlignTTS [19] for testing.”
• Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling (2020, methodology): “Finally, AlignTTS (Zeng et al., 2020) forgoes target durations completely and uses a specialized alignment loss inspired by the Baum-Welch algorithm to train a mixture density network for alignment.”
• SpeechNet: A Universal Modularized Model for Speech Processing Tasks (2021, unknown): “…et al., 2018; Karita et al., 2019), SE (Kim et al., 2020; Fu et al., 2020; Nicolson and Paliwal, 2020), SC (Katta et al., 2020; Safari and Hernando, 2020; Shi et al., 2020), TTS (Ren et al., 2020; Li et al., 2019; Zeng et al., 2020) and VC (Huang et al., 2020; Liu et al., 2020b; Lin et al., 2020).”
• Noise Level Limited Sub-Modeling for Diffusion Probabilistic Vocoders (2021, methodology): “The feedforward Transformer-based duration predictor used in [32] was modified for full-context label input and introduced.”
• DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs (2022, background): “They either leverage an external text-to-acoustic alignment module (Ren et al., 2019; 2021a; Peng et al., 2020; Elias et al., 2021) or jointly train one within the TTS model (Zeng et al., 2020; Miao et al., 2021; Badlani et al., 2021).”
• Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, methodology): “On the contrary, NAR models [12-21] achieve extremely fast inference speed, where a length regulator is used to replace the attention module in AR models to predict phoneme durations and expand the phoneme embeddings for parallel generation.”
• JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment (2020, background): “More recently, duration informed network not requiring pre-trained model [11] has been proposed, but it still requires a multi-stage training phase.”