Going deeper in facial expression recognition using deep neural networks

By Ali Mollahosseini, David Chan, Mohammad H. Mahoor. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2015

963 citations recorded by the source corpus.

Automated Facial Expression Recognition (FER) has remained a challenging and interesting problem in computer vision. Despite efforts made in developing various methods for FER, existing approaches lack generalizability when applied to unseen images or those that are captured in wild setting (i.e. the results are not significant). Most of the existing approaches are based on engineered features (e.g. HOG, LBPH, and Gabor) where the classifier's hyper-parameters are tuned to give best recognition accuracies across a single database, or a small collection of similar databases. This paper proposes a deep neural network architecture to address the FER problem across multiple well-known standard face datasets. Specifically, our network consists of two convolutional layers each followed by max pooling and then four Inception layers. The network is a single component architecture that takes registered facial images as the input and classifies them into either of the six basic or the neutral expressions. We conducted comprehensive experiments on seven publicly available facial expression databases, viz. MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER2013. The results of our proposed architecture are comparable to or better than the state-of-the-art methods and better than traditional convolutional neural networks in both accuracy and training time.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/83eba2f478a7e0cef6979f275c9c960b859bf99a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Citations in the corpus

Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks (2020, background): “Sigmoid and Tanh Functions Comparison Diagram Mollahosseini, Chan, and Mahoor[23] suggest that the hyperbolic function is sigmoidal, just like in the case of sigmoid functions.”
Spatial–Temporal Recurrent Neural Network for Emotion Recognition (2017, methodology): “, TMS [36], Cbin [14], and DNN [16], adopt different experimental settings by either taking static images as input or using different cross validation protocols.”
From Facial Expression Recognition to Interpersonal Relation Prediction (2016, background): “Social signal processing mainly involves facial expression recognition (Zhao and Pietikainen 2007; Tian et al. 2011; Liu et al. 2014b; Ruiz et al. 2015; Wu and Ji 2016; Fabian Benitez-Quiroz et al. 2016; Liu et al. 2014a, c; Jung et al. 2015;Mollahosseini et al. 2016; Fabian Benitez-Quiroz et al. 20”
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism (2019, methodology): “[49] adopted an inception based CNN and provided the average cross-dataset recognition accuracy.”
Attention mechanism-based CNN for facial expression recognition (2020, methodology): “When classifying static images, a facial expression recognition network [4] inspired by GoogLeNet was proposed.”
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild (2017, result): “In [31], a DCNN with inception layers was proposed to gain comparable results.”
A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analytics (2017, background): “Recently, deep learning has been demonstrated to be effective in solving this problem [28, 32].”
A Brief Review of Facial Emotion Recognition Based on Visual Information (2018, background): “Among several deep-learning-based approaches, two have a relatively higher performance compared to several state-of-the-art methods; a complex CNN network proposed in [72] consists of two convolutional layers, each followed by max pooling and four Inception layers.”
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild (2017, methodology): “[43] CK+ MultiPie - Inception based Convolutional Neural Network (CNN) - Subject-independent and cross-database experiments - 93.”
FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition (2016, result): “Compared with networks adopted in previous works [9], [23], [25], AlexNet [47] or VGG-M [48], the size of our network is fairly small.”
Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network (2019, methodology): “Mollahosseini [9] proposed a neural network for FER using two convolution layers, one max pooling layer, and four “inception” layers, i.”
Modeling Stylized Character Expressions via Deep Learning (2016, background): “In the current practice of facial expression analysis, CNNs have shown the capability to learn the features that statistically allow the network to make the correct classification of the input data in various ways [29,30].”
Deep Facial Expression Recognition: A Survey (2018, methodology): “dure to enable feature learning, further face alignment using the coordinates of localized landmarks can substantially enhance the FER performance [14].”