Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS

By Rui Liu, Berrak Sisman, Feilong Bao, Guanglai Gao, Haizhou Li. 2020

22 citations recorded by the source corpus.

Tacotron-based end-to-end speech synthesis has shown remarkable voice quality. However, the rendering of prosody in the synthesized speech remains to be improved, especially for long sentences, where prosodic phrasing errors can occur frequently. In this letter, we extend the Tacotron-based speech synthesis framework to explicitly model the prosodic phrase breaks. We propose a multi-task learning scheme for Tacotron training, that optimizes the system to predict both Mel spectrum and phrase breaks. To our best knowledge, this is the first implementation of multi-task learning for Tacotron based TTS with a prosodic phrasing model. Experiments show that our proposed training scheme consistently improves the voice quality for both Chinese and Mongolian systems.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Yi Ren
7. Zhou Zhao
8. Lei He

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Although neural TTS models simplify the text analysis module, some features for prosody prediction are incorporated into text encoder, such as the prediction of pitch [292], duration [290], phrase break [206], breath or filled pauses [404] are built on top of the text (character or phoneme) encoder ”