Emotional Voice Conversion: Theory, Databases and ESD

By Kun Zhou, Berrak Sisman, Rui Liu, Haizhou Li. 2021

290 citations recorded by the source corpus.

In this paper, we first provide a review of the state-of-the-art emotional voice conversion research, and the existing emotional speech databases. We then motivate the development of a novel emotional speech database (ESD) that addresses the increasing research need. With this paper, the ESD database is now made available to the research community. The ESD database consists of 350 parallel utterances spoken by 10 native English and 10 native Chinese speakers and covers 5 emotion categories (neutral, happy, angry, sad and surprise). More than 29 hours of speech data were recorded in a controlled acoustic environment. The database is suitable for multi-speaker and cross-lingual emotional voice conversion studies. As case studies, we implement several state-of-the-art emotional voice conversion systems on the ESD database. This paper provides a reference study on ESD in conjunction with its release.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Signal Processing

Citations in the corpus

Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models (2022, methodology): “We upload audio samples used in experiments including additional samples on ESD [24] dataset at demo page (https://nardien.github.io/grad-stylespeech-demo).”
Emotion-Aware Prosodic Phrasing for Expressive Text-to-Speech (2023, methodology): “We first conduct objective observations on ESD dataset [18] to validate the strong correlation between emotion and prosodic phrasing.”
PromptTTS 2: Describing and Generating Voices with Text Prompt (2023, background): “Each point in figure stands for a speech and the speech with the same speaker or the same emotion (Zhou et al., 2021; 2022) has the same color.”
An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era (2022, background): “Our overview, thus, fills the gap between recent surveys in deep speech synthesis, which focuses on “mere” TTS [15], and older affective speech synthesis reviews that have become largely obsolete in the deep learning era [12], [16] or newer ones that are more limited in scope [17], [18].”
Textless Speech Emotion Conversion using Decomposed and Discrete Representations (2021, background): “There is an increasing number of studies for speech emotion conversion recently (Zhou et al., 2021c).”