Maximizing Mutual Information for Tacotron

By Peng Liu, Xixin Wu, Shiyin Kang, Guangzhi Li, Dan Su, Dong Yu. 2019

16 citations recorded by the source corpus.

End-to-end speech synthesis methods already achieve close-to-human quality performance. However compared to HMM-based and NN-based frame-to-frame regression methods, they are prone to some synthesis errors, such as missing or repeating words and incomplete synthesis. We attribute the comparatively high utterance error rate to the local information preference of conditional autoregressive models, and the ill-posed training objective of the model, which describes mostly the training status of the autoregressive module, but rarely that of the condition module. Inspired by InfoGAN, we propose to maximize the mutual information between the text condition and the predicted acoustic features to strengthen the dependency between them for CAR speech synthesis model, which would alleviate the local information preference issue and reduce the utterance error rate. The training objective of maximizing mutual information can be considered as a metric of the dependency between the autoregressive module and the condition module. Experiment results show that our method can reduce the utterance error rate.

Subfields: Artificial Intelligence, Speech & Audio, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Xinyu Yang
7. Zhaoxi Mu
8. Yizhuo Dong

In Speech & Audio

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Zhaoxi Mu
6. Xinyu Yang
7. Yizhuo Dong
8. Soroosh Mariooryad

In Information Theory

1. Jaesam Yoon
2. Won Jang
3. Bongwan Kim
4. D. Lim

Citations in the corpus

• You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation (2020, background): “3], and CTC loss between spectrogram and input text (following [28]).”
• A Survey on Neural Speech Synthesis (2021, methodology): “[199] leverage CTC [91] based ASR as a cycle loss to encourage the generated mel-spectrograms to contain text information, which can also enhance the encoder-decoder connection for better attention alignment.”
• Location-Relative Attention Mechanisms for Robust Long-Form Speech Synthesis (2019, background): “Despite these successes, there are sometimes complaints of a lack of robustness in the alignment procedure that leads to missing or repeating words, incomplete synthesis, or an inability to generalize to longer utterances [5, 6, 7].”
• Review of end-to-end speech synthesis technology based on deep learning (2021, methodology): “[124] learned from the idea of InfoGAN [28] and proposed to use an additional auxiliary CTC recognizer to recognize the predicted acoustic features.”
• JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment (2020, background): “In [14], authors insist that the synthesis error rate of attention based TTS models can be alleviated if the model is guided to learn the dependency between the input text and the predicted acoustic feature sequences by maximizing the mutual information between them.”