Emotion recognition using deep learning approach from audio-visual emotional big data

By M. Shamim Hossain, G. Muhammad. Information Fusion, 2019

397 citations recorded by the source corpus.

Abstract This paper proposes an emotion recognition system using a deep learning approach from emotional Big Data. The Big Data comprises of speech and video. In the proposed system, a speech signal is first processed in the frequency domain to obtain a Mel-spectrogram, which can be treated as an image. Then this Mel-spectrogram is fed to a convolutional neural network (CNN). For video signals, some representative frames from a video segment are extracted and fed to the CNN. The outputs of the two CNNs are fused using two consecutive extreme learning machines (ELMs). The output of the fusion is given to a support vector machine (SVM) for final classification of the emotions. The proposed system is evaluated using two audio–visual emotional databases, one of which is Big Data. Experimental results confirm the effectiveness of the proposed system involving the CNNs and the ELMs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/d0a739df64bcee8008f75b489ed14a3f6214e175

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shuyong Gao
2. Weizhen Song
3. D. Bull
4. Nantheera Anantrasirichai
5. Antonio Liotta
6. Xinlei Li
7. Da-Wei Yang
8. Wei Tao

In Computer Vision and Pattern Recognition

1. D. Bull
2. Nantheera Anantrasirichai
3. Antonio Liotta
4. Weifeng Ge
5. Gyanendra Kumar Verma
6. Anamika Dhillon

In Experimental and Cognitive Psychology

1. D. Bull
2. Nantheera Anantrasirichai
3. Gyanendra Kumar Verma

Citations in the corpus

• Convolutional neural network: a review of models, methodologies and applications to object detection (2019, background): “Ghulam Muhammad [73] 2019 DNN, Faster R-CNN VOC Increases 25.”
• A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, methodology): “Model-level fusion, discovers the correlation properties between features extracted from different modalities, and uses or designs the fusion model with relaxed and smooth types such as HMM and two-stage ELM [353].”
• Artificial intelligence in the creative industries: a review (2020, background): “eyebrows, the tip of nose, the corners of mouth) [280], and both audio and video [281].”