An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
By Andreas Triantafyllopoulos, Björn Schuller, Gokcce .Iymen, M. Sezgin, Xiangheng He, Zijiang Yang, Panagiotis Tzirakis, Shuo Liu, Silvan Mertes, Elisabeth Andr'e and 2 more. 2022
105 citations recorded by the source corpus.
Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human–computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. However, the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions—aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesized utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology that underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In this overview, we outline ongoing trends and summarize state-of-the-art approaches in an attempt to provide a broad overview of this exciting field.
Subfields: Artificial Intelligence, Experimental and Cognitive Psychology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sri Karlapati
- Alexis Moinet
- Thomas Drugman
- Bartosz Putrycz
- Ewa Muszynska
- Mateusz Lajszczak
- Zhenhua Ling
- Yang Ai
In Experimental and Cognitive Psychology
Citations in the corpus
- BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (2024, background): “…that this also holds for LTTS, we propose an evaluation scheme to assess potential emergent abilities in TTS, identifying seven categories that are challenging from the literature [27–31]: compound nouns, emotions, foreign words, paralinguistics, punctuations, questions, and syntactic complexities.”