Sequence-to-sequence Modelling of F0 for Speech Emotion Conversion

By Carl Robinson, Nicolas Obin, Axel Röbel. 2019

50 citations recorded by the source corpus.

Voice interfaces are becoming wildly popular and driving demand for more advanced speech synthesis and voice transformation systems. Current text-to-speech methods produce realistic sounding voices, but they lack the emotional expressivity that listeners expect, given the context of the interaction and the phrase being spoken. Emotional voice conversion is a research domain concerned with generating expressive speech from neutral synthesised speech or natural human voice. This research investigated the effectiveness of using a sequence-to-sequence (seq2seq) encoder-decoder based model to transform the intonation of a human voice from neutral to expressive speech, with some preliminary introduction of linguistic conditioning. A subjective experiment conducted on the task of speech emotion recognition by listeners successfully demonstrated the effectiveness of the proposed sequence-to-sequence models to produce convincing voice emotion transformations. In particular, conditioning the model on the position of the syllable in the phrase significantly improved recognition rates.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale (2023, background): “Voice conversion [27, 39], emotion conversion [53, 34], speech enhancement [63, 11, 55] belong to this category.”
Textless Speech Emotion Conversion using Decomposed and Discrete Representations (2021, methodology): “Another line of works consists of modifying the parameters of the speech signal (Polyak et al., 2020b,a; Sisman et al., 2020; Robinson et al., 2019; Zhou et al., 2021b), i.”
An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era (2022, methodology): “Finally, some works evaluate ESS approaches by how well annotators are able to classify the synthesized emotions [49], [117], [122], [124], [125], [127].”