Multimodal Continuous Emotion Recognition with Data Augmentation Using Recurrent Neural Networks
By Jian Huang, Ya Li, Jianhua Tao, Zheng Lian, Mingyue Niu, Minghao Yang. 2018
35 citations recorded by the source corpus.
This paper presents our effects for Cross-cultural Emotion Sub-challenge in the Audio/Visual Emotion Challenge (AVEC) 2018, whose goal is to predict the level of three emotional dimensions time-continuously in a cross-cultural setup. We extract the emotional features from audio, visual and textual modalities. The state of art regressor for continuous emotion recognition, long short term memory recurrent neural network (LSTM-RNN) is utilized. We augment the training data by replacing the original training samples with shorter overlapping samples extracted from them, thus multiplying the number of training samples and also beneficial to train emotional temporal model with LSTM-RNN. In addition, two strategies are explored to decrease the interlocutor influence to improve the performance. We also compare the performance of feature level fusion and decision level fusion. The experimental results show the efficiency of the proposed method and competitive results are obtained.
Subfields: Experimental and Cognitive Psychology, Multimodal Learning, Sentiment & Opinion Analysis, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Experimental and Cognitive Psychology
- Maja Pantic
- Björn Schuller
- Michel François Valstar
- R. Cowie
- Mohammad Soleymani
- Fabien Ringeval
- Hatice Gunes
- Stefanos P. Zafeiriou
In Multimodal Learning
In Sentiment & Opinion Analysis
In Signal Processing
- Maja Pantic
- Michel François Valstar
- Björn Schuller
- R. Cowie
- Fabien Ringeval
- Mohammad Soleymani
- Maximilian Schmitt
- Soujanya Poria
Citations in the corpus
- AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition (2019, result): “nt detection to identify emotion markers invariant to socioculturalcontext. The other two entrants employed data driven approaches based on long short-term memory recurrent neural networks (LSTM-RNN) [27, 82]. Matching with similar results in the literature [21, 56], all entrants in the AVEC 2018 CE”