Automatic facial expression recognition using features of salient facial patches

By S. Happy, Aurobinda Routray. IEEE Transactions on Affective Computing, 2015

496 citations recorded by the source corpus.

Extraction of discriminative features from salient facial patches plays a vital role in effective facial expression recognition. The accurate detection of facial landmarks improves the localization of the salient patches on face images. This paper proposes a novel framework for expression recognition by using appearance features of selected facial patches. A few prominent facial patches, depending on the position of facial landmarks, are extracted which are active during emotion elicitation. These active patches are further processed to obtain the salient patches which contain discriminative features for classification of each pair of expressions, thereby selecting different facial patches as salient for different pair of expression classes. One-against-one classification method is adopted using these features. In addition, an automated learning-free facial landmark detection technique has been proposed, which achieves similar performances as that of other state-of-art landmark detection methods, yet requires significantly less execution time. The proposed method is found to perform well consistently in different resolutions, hence, providing a solution for expression recognition in low resolution images. Experiments on CK+ and JAFFE facial expression databases show the effectiveness of the proposed system.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/161eb88031f382e6a1d630cd9a1b9c4bc6b47652

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Shutao Li
2. Xiaojiang Peng
3. Yu Qiao
4. Guo-qing Guo
5. Fuyan Ma
6. Bin Sun
7. Luc Van Gool
8. Kai-Hong Wang

In Experimental and Cognitive Psychology

1. Fuyan Ma
2. Shutao Li
3. Bin Sun
4. Yu Qiao
5. Xiaojiang Peng
6. Jianfei Yang
7. Kai-Hong Wang
8. Debin Meng

Citations in the corpus

• Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network (2019, unknown): “2% Salient Facial Patch [46] 91.”
• Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion (2021, background): “Most of the researchers design their methods for occlusion-aware [21], [22] and multi-view FER, which are two main obstacles for FER in real-world scenarios.”
• TransFER: Learning Relation-aware Facial Expression Representations with Transformers (2021, background): “[30, 15, 31] extracted features on facial parts which are cropped around landmarks.”
• A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition (2017, background): “Happy and Routray [32] proposed a facial expression recognition system with the consideration of texture features of selected salient facial patches.”
• Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition (2019, background): “Subsequently, a number of works handle FER with sub-region based features and fusion schemes [22], [61], [62], which detect the occlusion regions first and then remove their local features.”