Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments

By Yusuke Yasuda, Xin Eric Wang, Junichi Yamagishi. 2019

8 citations recorded by the source corpus.

End-to-end text-to-speech (TTS) synthesis is a method that directly converts input text to output acoustic features using a single network. A recent advance of end-to-end TTS is due to a key technique called attention mechanisms, and all successful methods proposed so far have been based on soft attention mechanisms. However, although network structures are becoming increasingly complex, end-to-end TTS systems with soft attention mechanisms may still fail to learn and to predict accurate alignment between the input and output. This may be because the soft attention mechanisms are too flexible. Therefore, we propose an approach that has more explicit but natural constraints suitable for speech signals to make alignment learning and prediction of end-to-end TTS systems more robust. The proposed system, with the constrained alignment scheme borrowed from segment-to-segment neural transduction (SSNT), directly calculates the joint probability of acoustic features and alignment given an input text. The alignment is designed to be hard and monotonically increase by considering the speech nature, and it is treated as a latent variable and marginalized during training. During prediction, both the alignment and acoustic features can be generated from the probabilistic distributions. The advantages of our approach are that we can simplify many modules for the soft attention and that we can train the end-to-end TTS model using a single likelihood function. As far as we know, our approach is the first end-to-end TTS without a soft attention mechanism.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Sheng Zhao
5. Yi Ren
6. Frank K. Soong
7. Zhou Zhao
8. Xinyu Yang

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Sheng Zhao
5. Yi Ren
6. Frank K. Soong
7. Zhaoxi Mu
8. Xinyu Yang

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, background): “For monotonic attention [282, 46, 104, 404, 341], the attention position is monotonically increasing, which also leverages the prior that the alignments between text and speech are monotonic.”
• MultiSpeech: Multi-Speaker Text to Speech with Transformer (2020, background): “Monotonic and diagonal alignments in the attention weights be-tween text and speech are critical to ensure the quality of synthesized speech [12, 15, 16, 7, 17, 18].”
• Review of end-to-end speech synthesis technology based on deep learning (2021, background): “[240] also proposed a hard monotonic attention mechanism.”