Controlling Emotion Strength with Relative Attribute for End-to-End Speech Synthesis

By Xiaolian Zhu, Shan Yang, Geng Yang, Lei Xie. 2019

57 citations recorded by the source corpus.

Recently, attention-based end-to-end speech synthesis has achieved superior performance compared to traditional speech synthesis models, and several approaches like global style tokens are proposed to explore the style controllability of the end-to-end model. Although the existing methods show good performance in style disentanglement and transfer, it is still unable to control the explicit emotion of generated speech. In this paper, we mainly focus on the subtle control of expressive speech synthesis, where the emotion category and strength can be easily controlled with a discrete emotional vector and a continuous simple scalar, respectively. The continuous strength controller is learned by a ranking function according to the relative attribute measured on an emotion dataset. Our method automatically learns the relationship between low-level acoustic features and high-level subtle emotion strength. Experiments show that our method can effectively improve the controllability for an expressive end-to-end model.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Rui Liu
2. Berrak Sisman
3. Haizhou Li
4. Lei Xie
5. Philip N. Garner
6. Bastian Schnell
7. Yinjiao Lei
8. Shan Yang

In Signal Processing

1. Rui Liu
2. Berrak Sisman
3. Haizhou Li
4. Lei Xie
5. Philip N. Garner
6. Bastian Schnell
7. Kun Zhou
8. Shan Yang

Citations in the corpus

• Fine-Grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis (2020, methodology): “In detail, we automatically learn the relative attributes of emotional speech compared to the neutral speech in the utterance level, where the attributes are proved to be strength-related [18].”
• Towards Multi-Scale Style Control for Expressive Speech Synthesis (2021, background): “For example, [11] employs a learnt ranking function [12] to extract local-scale emotion intensity scalars of each phoneme in the utterance.”
• Controllable Emotion Transfer For End-to-End Speech Synthesis (2020, unknown): “We use the same dataset as in [27]: a high-quality emotional speech corpus containing 14-hour of recordings by a professional Chinese actress.”
• Emotional Voice Conversion: Theory, Databases and ESD (2021, background): “Due to the lack of data, researchers mostly use internal databases for emotional text-to-speech studies (Um et al., 2020; Lei et al., 2021; Kwon et al., 2019; Zhu et al., 2019).”
• Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis (2022, background): “cn/opencpop/ 16], emotional speech [17, 18, 19, 20, 21, 22], and also singing voice [23, 24].”
• Improving Emotional TTS with an Emotion Intensity Input from Unsupervised Extraction (2021, methodology): “Recent work [9, 11] has used attribute ranks [10] to compute emotion intensities.”