Unsupervised Word-Level Prosody Tagging for Controllable Speech Synthesis

By Yiwei Guo, Chenpeng Du, Kai Yu. 2022

16 citations recorded by the source corpus.

Although word-level prosody modeling in neural text-to-speech (TTS) has been investigated in recent research for diverse speech synthesis, it is still challenging to control speech synthesis manually without a specific reference. This is largely due to lack of word-level prosody tags. In this work, we propose a novel approach for unsupervised word-level prosody tagging with two stages, where we first group the words into different types with a decision tree according to their phonetic content and then cluster the prosodies using GMM within each type of words separately. This design is based on the assumption that the prosodies of different type of words, such as long or short words, should be tagged with different label sets. Furthermore, a TTS system with the derived word-level prosody tags is trained for controllable speech synthesis. Experiments on LJSpeech show that the TTS model trained with word-level prosody tags not only achieves better naturalness than a typical FastSpeech2 model, but also gains the ability to manipulate word-level prosody.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Chenpeng Du
2. Yiwei Guo
3. K. Yu
4. Xie Chen
5. Yichong Leng
6. Xuejiao Tan
7. Zhifang Guo
8. Sheng Zhao

Citations in the corpus

• VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature (2022, methodology): “For example, [11, 12, 13] uses phoneme-level prosodies, [14] uses word-level prosodies and [15, 16] uses hierarchical prosodies.”
• Prompttts: Controllable Text-To-Speech With Text Descriptions (2022, background): “Previous works on controllable TTS focus on controlling specific style factors: prosody control with word-level prosody tags [9], speaking speed control with sentence-level speaking-rate [10], and”