Investigating different representations for modeling and controlling multiple emotions in DNN-based speech synthesis

By Jaime Lorenzo-Trueba, Gustav Eje Henter, Shinji Takaki, Junichi Yamagishi, Yosuke Morino, Yuta Ochiai. 2018

93 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Wei-Ning Hsu
2. Xu Tan
3. Evgeny Kharitonov
4. Adam Polyak
5. Tu Nguyen
6. Jade Copet
7. Yossi Adi
8. Xin Eric Wang

Citations in the corpus

• Fine-Grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis (2020, methodology): “The most straightforward way to conduct emotional speech synthesis is using explicit annotations or labels as global render to model expressions [19, 20].”
• Model architectures to extrapolate emotional expressions in DNN-based text-to-speech (2021, methodology): “To control emotional strength, a method that used an auxiliary vector based on listener perception was proposed by [12].”
• Improving Emotional TTS with an Emotion Intensity Input from Unsupervised Extraction (2021, methodology): “Instead we rely on a classical RNN-based network, which has also been used in recent studies on emotional speech synthesis [27].”
• An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era (2022, methodology): “Parallel data approaches rely on datasets where the same speaker(s) have recorded a set of sentences by acting the entire set of different emotions [110], [113], [120], [136]—similar to parallel VC approaches.”
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis (2018, methodology): “The only difference from [63] is that the system was optimised using Adam [92] rather than stochastic gradient descent.”
• Textless Speech Emotion Conversion using Decomposed and Discrete Representations (2021, methodology): “This include using multilayer perceptron (Luo et al., 2016; Lorenzo-Trueba et al., 2018), highway neural network (Shankar et al.”
• Using Vaes and Normalizing Flows for One-Shot Text-To-Speech Synthesis of Expressive Speech (2019, unknown): “Existing work on emotional speech focuses on explicit emotional speech models [8], model adaptation [9] or voice conversion [10, 11], which again require significantly more data than the single utterance required by our proposed system.”
• Predicting Expressive Speaking Style from Text in End-To-End Speech Synthesis (2018, background): “Similarly, substantial effort has gone into modeling emotion, but has traditionally required keywords, semantic representations, or labels for model training; recent examples include [11], [12], [13], [14], and [15], [16]).”
• EMPHASIS: An Emotional Phoneme-based Acoustic Model for Speech Synthesis System (2018, methodology): “ANNbased method is also being applied for emotional speech synthesis and achieves natural emotional expressions [7], [8].”
• Nonparallel Expressive TTS for Unseen Target Speaker using Style-Controlled Adaptive Layer and Optimized Pitch Embedding (2023, methodology): “The style label is then used as conditional information to guide the TTS model in producing speech that matches the specific style category [5] [6].”