AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild

By Ali Mollahosseini, Behzad Hasani, Mohammad H. Mahoor. 2017

2,209 citations recorded by the source corpus.

Automated affective computing in the wild setting is a challenging problem in computer vision. Existing annotated databases of facial expressions in the wild are small and mostly cover discrete emotions (aka the categorical model). There are very limited annotated facial databases for affective computing in the continuous dimensional model (e.g., valence and arousal). To meet this need, we collected, annotated, and prepared for public distribution a new database of facial emotions in the wild (called AffectNet). AffectNet contains more than 1,000,000 facial images from the Internet by querying three major search engines using 1,250 emotion related keywords in six different languages. About half of the retrieved images were manually annotated for the presence of seven discrete facial expressions and the intensity of valence and arousal. AffectNet is by far the largest database of facial expression, valence, and arousal in the wild enabling research in automated facial expression recognition in two different emotion models. Two baseline deep neural networks are used to classify images in the categorical model and predict the intensity of valence and arousal. Various evaluation metrics show that our deep neural network baselines can perform better than conventional machine learning methods and off-the-shelf facial expression recognition systems.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Citations in the corpus

Compacting, Picking and Growing for Unforgetting Continual Learning (2019, methodology): “The AffectNet dataset [25] is used for expression classification that classifies faces into seven primary emotions.”
Persuasive Faces: Generating Faces in Advertisements (2018, methodology): “Facial expression and emotion recognition is an established and popular topic [4, 15, 23, 28, 32].”
EMOCA: Emotion Driven Monocular Face Capture and Animation (2022, methodology): “The table summarizes the effect of EMOCA trained w/ and w/o the emotion consistency loss, and using the DECA data only [14, 17] instead of the AffectNet training data [57].”
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism (2019, methodology): “1) Datasets: We evaluated the methods on both in-the-wild datasets (RAF-DB [4], AffectNet [5], SFEW [35]) and inthe-lab datasets (CK+ [1], MMI [2], and Oulu-CASIA [3]).”
Factorized Higher-Order CNNs With an Application to Spatio-Temporal Emotion Estimation (2019, background): “As a result, most work in this area of affect estimation in-the-wild focuses on affect estimation from static imagery [23, 30].”
Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace (2019, background): “In [29] a static in-the-wild database (AffectNet) has been created that contains VA annotations for 1 M images.”
Comparing Recognition Performance and Robustness of Multimodal Deep Learning Models for Multimodal Emotion Recognition (2021, background): “Therefore, recognizing emotions with multiple modalities has become a promising method [20]–[23].”
Increasingly Packing Multiple Facial-Informatics Modules in A Unified Deep-Learning Model via Lifelong Learning (2019, methodology): “In facial expression, the AffectNet [16] is a famous baseline in recent research and it also releases the largest facial expression dataset.”
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition (2019, methodology): “[21], [48], [36], [17].”
Learning Disentangled Representations with Reference-Based Variational Autoencoders (2019, positive): “As unlabelled images we use a subset of the AffectNet dataset (Mollahosseini et al., 2017), which contains a large quantity of facial images.”
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition (2019, background): “AffectNet [53] contains more than one million images obtained from the Internet by querying different search engines using emotion related tags.”
Robust Lightweight Facial Expression Recognition Network with Label Distribution Training (2021, background): “For AffectNet, some work veriﬁes their methods on AffectNet-7 but others on AffectNet-8, while we conduct experiments and compare with other state-of-the-art methods both on AffectNet-7 and AffectNet-8.”
TransFER: Learning Relation-aware Facial Expression Representations with Transformers (2021, background): “AffectNet [20] is the largest publicly available FER dataset so far.”
Deep Facial Expression Recognition: A Survey (2018, methodology): “A reasonable approach is to employ crowd-sourcing models [34], [45], [208] under the guidance”
IExpressNet: Facial Expression Recognition with Incremental Classes (2020, background): “AffectNet [47] contains about 450,000 images, each of which is manually annotated as seven expression classes by one Poster Session D2: Emerging Multimedia Applications & Emotional and Social Signals in Multimedia annotator.”
Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion (2021, background): “However, the performance degrades dramatically on the real-world FER datasets, such as RAF-DB[8], FERPlus[15] and AffectNet[16].”
Personalized machine learning for robot perception of affect and engagement in autism therapy (2018, background): “The state of the art for recognition of arousal and valence benchmarked on the largest public data set of human facial expressions [Affect-Net (43)] claims a human labeler agreement of 60.”