Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network

By George Trigeorgis, Fabien Ringeval, Raymond Brueckner, Erik Marchi, Mihalis A. Nicolaou, Björn Schuller, Stefanos P. Zafeiriou. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2016

895 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Experimental and Cognitive Psychology, Signal Processing

View Paper: https://www.semanticscholar.org/paper/96d27c4e487de191a17476a9775a51678a879d18

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Experimental and Cognitive Psychology

In Signal Processing

Citations in the corpus

Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph (2018, methodology): “For acoustic only baseline models: AdieuNet (Trigeorgis et al., 2016), SER-LSTM (Lim et al., 2016).”
A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition (2019, methodology): “[24] presented a technique for spontaneous SER-based on CNN and LSTM using the REmote COLlaborative and Affective RECOLA natural emotion database.”
Interpretable Convolutional Filters with SincNet (2018, background): “This approach has shown promise in speech [25, 26, 27, 28, 29], including emotion tasks [30], speaker recognition [35], spoofing detection [34], and speech synthesis [31, 32].”
AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition (2019, methodology): “The predictions of all test sequences from each culture are concatenated prior to computing the CCC , whose opposite is used as loss function for training the networks [71, 77].”
Tensor Fusion Network for Multimodal Sentiment Analysis (2017, methodology): “Adieu-Net (Trigeorgis et al., 2016) is an end-to-end approach for emotion recognition in audio using directly PCM features.”
Multimodal Machine Learning: A Survey and Taxonomy (2017, background): “Similarly, in the audio domain, acoustic features such as Mel-frequency cepstral coefficients (MFCC) have been superseded by data-driven deep neural networks in speech recognition [79] and recurrent neural networks for para-linguistic analysis [207].”
Learning Speaker Representations with Mutual Information (2018, positive): “Similarly to [25–29], we propose to directly process raw waveforms rather than using standard MFCC, or FBANK features.”
Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond (2018, methodology): “Our loss function was based on the Concordance Correlation Coefficient (CCC) metric that has been shown to provide better insight on whether the prediction follows the structure of the ground truth annotation [54] [53].”
3-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition (2018, background): “In recent years, CNN has exhibited excellent performances in the field of SER [1], [4], [6].”
The INTERSPEECH 2020 Computational Paralinguistics Challenge: Elderly Emotion, Breathing & Masks (2020, background): “This architecture has been successful in the task of sequential emotion recognition [35, 36].”
Speech Emotion Recognition Using Deep Learning Techniques: A Review (2019, background): “there is a shift to semi supervised learning [90].”
Deep Audio-visual Learning: A Survey (2020, background): “the limited information in the visual modality may lead to a performance bottleneck. To combine both audio and visual information for various scenes, especially in noisy conditions, Trigeorgis et al. [60] introduced an end-to-end model to obtain a ‘context-aware’ feature from the raw temporal repres”
End-to-End Multimodal Emotion Recognition Using Deep Neural Networks (2017, methodology): “[10] proposed an end-to-end model that uses a CNN to extract features from the raw signal and then an LSTM network to capture the contextual information in the data.”
Speech Emotion Recognition Using Deep Convolutional Neural Network and Discriminant Temporal Pyramid Matching (2018, methodology): “Recently, Trigeorgis et al., [17] presents an end-to-end speech emotion recognition system by combining a 2-layer CNN with a Long Short-Term Memory (LSTM) [18].”