Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems

By Takuma Okamoto, Tomoki Toda, Yoshinori Shiga, Hisashi Kawai. 2019

26 citations recorded by the source corpus.

Although sequence-to-sequence (seq2seq) models with attention mechanism in neural text-to-speech (TTS) systems, such as Tacotron 2, can jointly optimize duration and acoustic models, and realize high-fidelity synthesis compared with conventional duration-acoustic pipeline models, these involve a risk that speech samples cannot be sometimes successfully synthesized due to the attention prediction errors. Therefore, these seq2seq models cannot be directly introduced in practical TTS systems. On the other hand, the conventional pipeline models are broadly used in practical TTS systems since there are few crucial prediction errors in the duration model. For realizing high-quality practical TTS systems without attention prediction errors, this paper investigates Tacotron-based acoustic models with phoneme alignment instead of attention. The phoneme durations are first obtained from HMM-based forced alignment and the duration model is a simple bidirectional LSTM-based network. Then, a seq2seq model with forced alignment instead of attention is investigated and an alternative model with Tacotron decoder and phoneme duration is proposed. The results of experiments with full-context label input using WaveGlow vocoder indicate that the proposed model can realize a high-fidelity TTS system for Japanese with a real-time factor of 0.13 using a GPU without attention prediction errors compared with the seq2seq models.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Ryuichi Yamamoto
6. Sheng Zhao
7. Jae-Min Kim
8. Eunwoo Song

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Ryuichi Yamamoto
6. Jae-Min Kim
7. Eunwoo Song
8. Min-Jae Hwang

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Replacing Attention with Duration Prediction Label from encoder-decoder attention [284, 355, 193, 177] Label from CTC alignment [19] Label from HMM alignment [286, 411, 190, 246, 72, 298] Dynamic programming [422, 189, 229] Monotonic alignment search [156] Monotonic interpolation with soft DTW [67, ”
• Improved Parallel Wavegan Vocoder with Perceptually Weighted Spectrogram Loss (2021, methodology): “Although there are many state-of-the-art acoustic architectures available [24–26], we used a Tacotron model with phoneme alignment [27, 28] for its fast and stable generation and competitive synthesis quality.”
• TTS-by-TTS: TTS-Driven Data Augmentation for Fast and High-Quality Speech Synthesis (2020, methodology): “Thus, we adopted a Tacotron 2 decoder with a phoneme alignment approach [6], which has the capacity to accurately align the phoneme sequence with the acoustic features, as an acoustic model.”
• Neural Text-to-Speech with a Modeling-by-Generation Excitation Vocoder (2020, methodology): “Although there are many state-of-the-art acoustic models available, including Tacotron and Transformer [11,14,20], we opted to pursue a Tacotron model with phoneme alignment approach [15] because of its fast and stable generation and competitive synthetic quality.”
• Noise Level Limited Sub-Modeling for Diffusion Probabilistic Vocoders (2021, methodology): “A total of 25,046 (18 h) and 20 utterances were respectively used for the training and test sets, as in [29, 30].”