Textless Speech Emotion Conversion using Decomposed and Discrete Representations

By Felix Kreuk, Adam Polyak, Jade Copet, Evgeny Kharitonov, Tu Nguyen, Morgane Rivière, Wei-Ning Hsu, Abdel-rahman Mohamed, Emmanuel Dupoux, Yossi Adi. 2021

42 citations recorded by the source corpus.

Subfields: Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Speech & Audio

1. Wei-Ning Hsu
2. Yossi Adi
3. Gabriel Synnaeve
4. Abdel-rahman Mohamed
5. Jade Copet
6. Jinyu Li
7. Shinji Watanabe
8. Ann Lee

Citations in the corpus

• High Fidelity Neural Audio Compression (2022, methodology): “Similar methods were applied to speech resynthesis (Polyak et al., 2021), speech emotion conversion (Kreuk et al., 2021), spoken dialog system (Nguyen et al., 2022), and speech-to-speech translation (Lee et al., 2021a;b; Popuri et al., 2022).”
• Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale (2023, background): “Many of these models are supervised and trained on pairs of data that only differ in one attribute, for example, emotion [34].”
• Textless Speech-to-Speech Translation on Real Data (2021, methodology): “We further reduce the full orig-unit sequence by removing repeating units (Lakhotia et al., 2021; Lee et al., 2021; Kharitonov et al., 2021; Kreuk et al., 2021).”
• Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision (2023, background): “At its core, SPEAR-TTS leverages recent advances in the “textless” modeling of spoken language (Lakhotia et al., 2021; Dunbar et al., 2021; Polyak et al., 2021; Kreuk et al., 2021; Kharitonov et al., 2022; Nguyen et al., 2022; Borsos et al., 2022).”
• AudioGen: Textually Guided Audio Generation (2022, background): “, 2021), speech emotion conversion (Kreuk et al., 2021), spoken dialuge (Nguyen et al.”
• Textually Pretrained Speech Language Models (2023, background): “Kreuk et al. [2021] proposed to cast the task of speech emotion conversion as a translation task, hence translating between one emotion to the other in the discrete space, while Maimon and Adi [2022] proposed a similar approach for speaking style conversion.”
• An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era (2022, methodology): “[115] Fixed N/A Frame 7 Transformation Seq2Seq Spectra/F0/T EVC”
• Self-Supervised Speech Representation Learning: A Review (2022, methodology): “speech representation learning to model language directly from audio, bypassing the need for text or labels [76], [77], [78], [331].”