Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings

By Leonardo Pepino, Pablo Ernesto Riera, Luciana Ferrer. 2021

483 citations recorded by the source corpus.

Emotion recognition datasets are relatively small, making the use of the more sophisticated deep learning approaches challenging. In this work, we propose a transfer learning method for speech emotion recognition where features extracted from pre-trained wav2vec 2.0 models are modeled using simple neural networks. We propose to combine the output of several layers from the pre-trained model using trainable weights which are learned jointly with the downstream model. Further, we compare performance using two different wav2vec 2.0 models, with and without finetuning for speech recognition. We evaluate our proposed approaches on two standard emotion databases IEMOCAP and RAVDESS, showing superior performance compared to results in the literature.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Sentiment & Opinion Analysis, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Jianfeng Gao
  4. Graham Neubig
  5. Percy Liang
  6. Yoshua Bengio
  7. Quoc V. Le
  8. Michael Auli

In Experimental and Cognitive Psychology

  1. Kyu J. Han
  2. Xie Chen
  3. Michael Auli
  4. Alexis Conneau
  5. Wei-Ning Hsu
  6. Yoav Artzi
  7. Alexei Baevski

In Speech & Audio

  1. Wei-Ning Hsu
  2. Alexis Conneau
  3. Alexei Baevski
  4. Michael Auli
  5. Xie Chen
  6. Kyu J. Han
  7. Yoav Artzi
  8. Kilian Q. Weinberger

In Signal Processing

  1. Wei-Ning Hsu
  2. Alexei Baevski
  3. Alexis Conneau
  4. Michael Auli
  5. Xie Chen
  6. Kyu J. Han
  7. Yoav Artzi
  8. Kilian Q. Weinberger

Citations in the corpus