Implementing Prosodic Phrasing in Chinese End-to-end Speech Synthesis

By Yanfeng Lu, Minghui Dong, Ying Chen. 2019

29 citations recorded by the source corpus.

Text-to-Speech (TTS) systems have been evolving rapidly in recent years. With the great modelling power of deep neural networks, researchers have achieved end-to-end conversion from raw text to speech. It has been shown by various research projects that end-to-end TTS systems are able to generate speech that sounds akin to human voice for English and other languages. However, for languages like Chinese, there are two problems to deal with. Firstly, due to the large character set, a small input set comparable to the English character set is needed for the end-to-end solution. Secondly, there are serious prosodic phrasing mistakes when the end-to-end method is applied to Chinese. In this paper, we will propose a solution for an end-to-end Chinese TTS system on the basis of Tacotron 2 and Wavenet vocoder. We will then add extra contextual information to improve the performance of prosodic phrasing. Our experiments have demonstrated the effectiveness of this proposal.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Lei He
6. Haizhou Li
7. Sheng Zhao
8. Berrak Sisman

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Lei He
6. Hisashi Kawai
7. Haizhou Li
8. Berrak Sisman

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “Some works [50, 3, 62, 322, 210, 212] investigate different model structures such as CRF [176], RNN [111], and self-attention [362] for prosody prediction in Chinese.”
• Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems (2019, methodology): “Additionally, a seq2seq model for Chinese has been improved using full-context label input [31].”
• A Unified Sequence-to-Sequence Front-End Model for Mandarin Text-to-Speech Synthesis (2019, result): “The prosody structure in Mandarin TTS determines the naturalness to a great extent, which has been proved in previous researches [5, 6].”
• Controlling Emotion Strength with Relative Attribute for End-to-End Speech Synthesis (2019, unknown): “, without explicit prosodic labels [7, 9, 12–14].”
• Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS (2020, background): “In [17], three different contextual information were selected to improve their Chinese TTS.”
• Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS (2020, background): “There have been recent studies on prosody modeling for end-toend TTS system [29], for example, to improve the prosodic phrasing by using contextual information [30], and syntactic features [31].”