Learning active facial patches for expression analysis

By Lin Zhong, Qingshan Liu, Peng Yang, Bo Liu, Junzhou Huang, Dimitris N. Metaxas. 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012

380 citations recorded by the source corpus.

In this paper, we present a new idea to analyze facial expression by exploring some common and specific information among different expressions. Inspired by the observation that only a few facial parts are active in expression disclosure (e.g., around mouth, eye), we try to discover the common and specific patches which are important to discriminate all the expressions and only a particular expression, respectively. A two-stage multi-task sparse learning (MTSL) framework is proposed to efficiently locate those discriminative patches. In the first stage MTSL, expression recognition tasks, each of which aims to find dominant patches for each expression, are combined to located common patches. Second, two related tasks, facial expression recognition and face verification tasks, are coupled to learn specific facial patches for individual expression. Extensive experiments validate the existence and significance of common and specific patches. Utilizing these learned patches, we achieve superior performances on expression recognition compared to the state-of-the-arts.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Signal Processing

View Paper: https://www.semanticscholar.org/paper/d785fcf71cb22f9c33473cba35f075c1f0f06ffc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

In Signal Processing

Citations in the corpus

Learning to Transfer: Transferring Latent Task Structures and Its Application to Person-Specific Facial Action Unit Detection (2015, methodology): “For example, [28] defined each task to relate to a different face region, and used MTL to find a sparse set of face regions capturing facial expressions.”
A Novel Space-Time Representation on the Positive Semidefinite Cone for Facial Expression Recognition (2017, neutral): “12 (A) CSPL [48] (10-fold) 89.”
From Facial Expression Recognition to Interpersonal Relation Prediction (2016, neutral): “Accuracy CSPL [90] 89.9% LBPSVM[62] 95.1% BDBN [41] 96.7% PPDN [89] 97.3% Zero-bias CNN [29] 98.3% Our Method 98.9% six prototypical expressions, i.e. , angry, happy, surprise, sad, disgust, fear, or a non-standard expression ( i.e. contempt).”
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism (2019, background): “Localizing and encoding the expression-related parts is of beneﬁt to recognize facial expression [12].”
Exploiting sparsity and co-occurrence structure for action unit recognition (2015, background): “Zhong et al. [31] used group sparsity over the image space for AU recognition, where the face image is divided into non-overlapping patches and grouped by their conceptual roles in making expressions.”
Modeling Stylized Character Expressions via Deep Learning (2016, methodology): “SVMs are most commonly used for FER tasks [18,34,35].”
Facial Expression Recognition via a Boosted Deep Belief Network (2014, methodology): “These features can be either hand-designed [32, 33, 34, 25, 1, 27, 28, 26, 22, 9, 4, 10, 13] or learned from training images [6, 29, 15, 16, 36, 2, 35, 30].”
Context-Aware Emotion Recognition Networks (2019, methodology): “Previous researches for emotion recognition based on handcrafted features [4, 5] or deep networks [6, 7, 8] have mainly focused on the perception of the facial expression, based on the assumption that facial images are one of the most discriminative features of emotional responses.”
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks (2017, methodology): “In this paper, we conduct our experiments on all of the 205 sequences with 10-fold cross-validation, which is the same as [6], [7], and [45].”
Robust Lightweight Facial Expression Recognition Network with Label Distribution Training (2021, background): “Because there is a lot of redundant information in global facial features (Zhong et al. 2012, 2014), the channel-spatial modulator is introduced to highlight the crucial global features after the Stage2.”
Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order (2017, methodology): “[68] did not report the training and recognition time.”
FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition (2016, background): “In [16], Zhong et al. observed that only a few active facial patches are useful for expression recognition.”
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition (2014, methodology): “HOE [27] 54.15 LBP-TOP [34] 59.51 HOG 3D [14] 60.89 3D SIFT [23] 64.39 ADL [24] 47.78 HMM [30] 51.5 ITBN [30] 59.7 CPL [35] 49.36 CSPL [35] 73.53 Ours STM 65.37 Ours STM-ExpLet 75.12 Table 2.”
Deep Facial Expression Recognition: A Survey (2018, background): “, local binary patterns (LBP) [12], LBP on three orthogonal planes (LBP-TOP) [15], non-negative matrix factorization (NMF) [19] and sparse learning [20]) for FER.”
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition (2015, methodology): “In this section, we compare our approach with other state-of-the-art algorithms in facial expression recognition, such as manifold-based sparse representation (MSR) [14], AdaLBP [23], Atlases [4], and common and specific active patches (CSPL) [25].”
Automatic facial expression recognition using features of salient facial patches (2015, methodology): “Performance is improved by the use of both geometric and appearance features.”
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition (2015, background): “The optimality of such techniques, however, is being questioned in the scope of affect recognition, and new trends address the importance of making use of domain knowledge explicitly when developing dimensionality reduction techniques [163], [189].”