Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition
By Shan Li, Weihong Deng. 2019
601 citations recorded by the source corpus.
Facial expression is central to human experience, but most previous databases and studies are limited to posed facial behavior under controlled conditions. In this paper, we present a novel facial expression database, Real-world Affective Face Database (RAF-DB), which contains approximately 30 000 facial images with uncontrolled poses and illumination from thousands of individuals of diverse ages and races. During the crowdsourcing annotation, each image is independently labeled by approximately 40 annotators. An expectation–maximization algorithm is developed to reliably estimate the emotion labels, which reveals that real-world faces often express compound or even mixture emotions. A cross-database study between RAF-DB and CK+ database further indicates that the action units of real-world emotions are much more diverse than, or even deviate from, those of laboratory-controlled emotions. To address the recognition of multi-modal expressions in the wild, we propose a new deep locality-preserving convolutional neural network (DLP-CNN) method that aims to enhance the discriminative power of deep features by preserving the locality closeness while maximizing the inter-class scatter. Benchmark experiments on 7-class basic expressions and 11-class compound expressions, as well as additional experiments on CK+, MMI, and SFEW 2.0 databases, show that the proposed DLP-CNN outperforms the state-of-the-art handcrafted features and deep learning-based methods for expression recognition in the wild. To promote further study, we have made the RAF database, benchmarks, and descriptor encodings publicly available to the research community.
Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Experimental and Cognitive Psychology
Citations in the corpus
- Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion (2021, methodology): “The methods in [8] presented their performance using mean accuracy.”
- Deep Facial Expression Recognition: A Survey (2018, background): “DB [34], [44] and AffectNet [45], and we anticipate that with advances in technology and the widespread of the Internet, more complementary facial expression datasets will be con-”
- Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition (2021, background): “In order to verify the efficacy of the individual loss functions proposed in our method, this paper now performs ablation studies on the RAF-DB dataset to demonstrate that our affinity loss and partition loss are indeed contributing to model performance.”
- Robust Lightweight Facial Expression Recognition Network with Label Distribution Training (2021, methodology): “To verify the effectiveness of the proposed method, we conduct the experiments on three popular in-the-wild facial expression datasets: RAF-DB (Li and Deng 2018), CAER-S (Lee et al. 2019), and AffectNet (Mollahosseini, Hasani, and Mahoor 2017), and five realistic occlusion and pose variation…”
- How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis (2020, background): “The only outlier in this regard was RAF-DB, which contained “5% remains unsure” in their description of gender annotations [94].”
- Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition (2019, methodology): “RAF-DB is a latest facial expression dataset which not only has basic emotion categories but also compound categories.”