3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition

By Mingyi Chen, Xuanji He, Jing Yang, Han Zhang. IEEE Signal Processing Letters, 2018

441 citations recorded by the source corpus.

Speech emotion recognition (SER) is a difficult task due to the complexity of emotions. The SER performances are heavily dependent on the effectiveness of emotional features extracted from the speech. However, most emotional features are sensitive to emotionally irrelevant factors, such as the speaker, speaking styles, and environment. In this letter, we assume that calculating the deltas and delta-deltas for personalized features not only preserves the effective emotional information but also reduces the influence of emotionally irrelevant factors, leading to reduce misclassification. In addition, SER often suffers from the silent frames and emotionally irrelevant frames. Meanwhile, attention mechanism has exhibited outstanding performances in learning relevant feature representations for specific tasks. Inspired by this, we propose a three-dimensional attention-based convolutional recurrent neural networks to learn discriminative features for SER, where the Mel-spectrogram with deltas and delta-deltas are used as input. Experiments on IEMOCAP and Emo-DB corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Signal Processing

View Paper: https://www.semanticscholar.org/paper/47c81f9c14234694619f8bd0ba5673e4e9af37b3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Signal Processing

Citations in the corpus

A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, methodology): “[199] 2018 Attention-3D-RNN FC IEMOCAP Emo-DB 4 classes: 64.”
Expressive TTS Training With Frame and Style Reconstruction Loss (2020, methodology): “D CNN for dimension reduction.”
A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition (2019, methodology): “Chen et al. [42] developed a system for SER using 3D CNN architecture and trained model to increase the accuracy of SER, but he also used the pooling scheme to develop the network.”
Survey on Deep Neural Networks in Speech and Vision Systems (2019, methodology): “This idea is later extended in Chen et al. [139] where operation on frequency bank representation of speech signals can be used as inputs into a convolutional layer.”