OpenFace: An open source facial behavior analysis toolkit

By Tadas Baltrusaitis, Peter Robinson, Louis-philippe Morency. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2016

1,776 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/d997919c30fa6711bc5c25cf8c8aea34fac27b91

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Citations in the corpus

Personalized machine learning for robot perception of affect and engagement in autism therapy (2018, methodology): “We extracted various features (the sensing step) from these modalities using state-of-the-art tools for video and audio processing (OpenFace ( 50 ), OpenPose ( 51 ), and OpenSmile ( 52 )) - see Appendix/C.”
Interpreting CNN Models for Apparent Personality Trait Regression (2017, methodology): “We used the same Openface library [2] applied to crop the faces and generate the face dataset used in Sections 4.”
Multimodal Continuous Emotion Recognition with Data Augmentation Using Recurrent Neural Networks (2018, methodology): “The geometric features are extracted by Openface [41], which is the first open source tool capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.”
Protecting World Leaders Against Deep Fakes (2019, methodology): “We use the open-source facial behavior analysis toolkit OpenFace2 [3, 2, 4] to extract facial and head movements in a video.”
Audio-Visual Fusion for Sentiment Classiﬁcation using Cross-Modal Autoencoder (2018, methodology): “The visual modality (V) is represented with features (100-dimensional) extracted, at clip-level, using FACET framework and OpenFace toolkit [28, 29].”
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis (2020, methodology): “For UR_FUNNY, OpenFace [4], a facial behavioral analysis tool, is used to extract features related to the facial expressions of the speaker.”
Every Smile is Unique: Landmark-Guided Diverse Smile Generation (2018, methodology): “Similar to the pre-processing steps of other works [46, 43], we also perform face alignment on the extracted sequences using OpenFace [5], aligning the faces with respect to the center of the two eyes horizontally and to the vertical line passing through the center of the two eyes and the mouth.”
AVEC 2017: Real-life Depression, and Affect Recognition Workshop and Challenge (2017, methodology): “Based on the OpenFace7 framework [2], we provide dierent types of video features8:”
Deep Multimodal Representation Learning: A Survey (2019, methodology): “For example, OpenFace [69] can be used to extract facial features such as facial landmark, head pose, and eye gaze.”
Multimodal sentiment analysis with word-level fusion and reinforcement learning (2017, methodology): “nts (MFCCs), pitch tracking and voiced/unvoiced segmenting features, glottal source parameters, peak slope parameters and maxima dispersion quotients. For video inputs, we use Facet [11] and OpenFace [4, 44] to extract a set of features including facial action units, facial landmarks, head pose, gaz”
Automatic Analysis of Facial Actions: A Survey (2019, background): “OpenFace [14] also constitutes an effort along these lines.”
OpenFace 2.0: Facial Behavior Analysis Toolkit (2018, methodology): “OpenFace 2.0 contains a direct implementation with a couple of changes that adapt it to work better on natural video sequences using person speciﬁc normalization and prediction correction [8], [11].”
EgoCom: A Multi-Person Multi-Modal Egocentric Communications Dataset (2020, background): “[28], and OpenFace, a multi-modal face recognition [29].”
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues (2020, methodology): “In particular, we use OpenFace [14] to extract 430-dimensional facial features, including the 2D landmarks positions, head pose orientation, and gaze features.”