Collecting Large, Richly Annotated Facial-Expression Databases from Movies

By Abhinav Dhall, Roland Göcke, Simon Lucey, Tom Gedeon. IEEE Multimedia, 2012

636 citations recorded by the source corpus.

Two large facial-expression databases depicting challenging real-world conditions were constructed using a semi-automatic approach via a recommender system based on subtitles.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/1336839e7bd3d072a2c699e613312a852a6586ae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Citations in the corpus

Recurrent Neural Networks for Emotion Recognition in Video (2015, background): “The Emotion recognition in the wild (EmotiW 2015) challenge [9] is an extension of a similar challenge held in 2014 [8].”
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition (2019, neutral): “The Static Facial Expressions in the Wild (SFEW) dataset is built by selecting frames from AFEW [15], which covers unconstrained facial expressions, varied head poses, large age range, occlusions, varied focus, different resolution of the face and real-world illumination.”
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild (2018, methodology): “utral, as discussed in Sec. 3), we note that it still contains some diversity of emotion. For comparison, we also illustrate the distribution of emotional responses of the teacher model on ‘Afew 6.0’ [25], an emotion recognition benchmark. The Afew dataset was collected by selecting scenes in movies”
A Novel Space-Time Representation on the Positive Semidefinite Cone for Facial Expression Recognition (2017, background): “AFEW database [12] – collected from movies showing close-to-real-world conditions, which depicts or simulates the spontaneous expressions in uncontrolled environment.”
Image based Static Facial Expression Recognition with Multiple Deep Network Learning (2015, background): “Categories and Subject Descriptors I.5.4 [Pattern Recognition]: Applications—computer vision, signal processing ; I.4.m [Image Processing and Computer Vision]: Miscellaneous Keywords Facial Expression Recognition; Convolutional Neural Network; Multiple Network Learning; EmotiW 2015 Challenge”
EmotioNet: An Accurate, Real-Time Algorithm for the Automatic Annotation of a Million Facial Expressions in the Wild (2016, background): “While small and medium size databases can be manually annotated by expert coders over several months [11, 5], large databases cannot.”
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism (2019, methodology): “SFEW The Static Facial Expressions in the Wild (SFEW) dataset [35] is created by selecting static frames from Acted Facial Expressions in the Wild (AFEW) [36].”
Video and Image based Emotion Recognition Challenges in the Wild: EmotiW 2015 (2015, background): “The details of database collection are discussed in [4].”
Deep Facial Expression Recognition: A Survey (2018, background): “*We included the result of a single spatiotemporal network and the best result after fusion with both video and audio modalities. y 7 Classes in CK+: anger, contempt, disgust, fear, happiness, sadness, and surprise. z 7 Classes in AFEW: anger, disgust, fear, happiness, neutral, sadness, and surprise”
Top-Down Neural Attention by Excitation Backprop (2016, methodology): “This model is trained on the training set of the Static Facial Expressions in the Wild (SFEW) dataset Dhall et al. (2012). Using the proposed c-MWP maps, we can visualize local evidence used by our model for a target emotion category.”
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition (2014, methodology): “The proposed method is evaluated on four public expression databases: CK+ [19], MMI [26], Oulu-CASIA VIS [33], AFEW [6], including both lab-controlled and wild scenarios.”
Developing crossmodal expression recognition based on a deep neural model (2016, methodology): “The third corpus is the EmotiW database, published by Dhall, Goecke, Lucey, and Gedeon (2012).”
Video-based emotion recognition using CNN-RNN and C3D hybrid networks (2016, methodology): “Additionally, for non-faces frames falsely detected by the face detector, a CNN-based face filter is applied for filtering out the mistakes.”
Context-Aware Emotion Recognition Networks (2019, methodology): “Comparison of the CAER with existing emotion recognition datasets such as EMOTIC [14], AffectNet [43], AFEW [44], and Video Emotion [45] datasets.”
Combining modality specific deep neural networks for emotion recognition in video (2013, background): “…video frames; (2) a deep belief net to capture audio information; (3) a deep autoencoder to model the spatiotemporal information produced by the human actions depicted within the entire scene; and (4) a shallow network architecture focused on extracted features of the mouth of the primary…”
Facial Expression Analysis under Partial Occlusion (2018, background): “5) The Acted Facial Expressions in the Wild (AFEW) database [Dhall, et al. 2012] is a dynamic temporal facial expressions data corpus that comprises of a total of 957 audio-visual clips extracted from 37 movies.”