3D Constrained Local Model for rigid and non-rigid facial tracking

By Tadas Baltrusaitis, Peter Robinson, Louis-philippe Morency. 2012

334 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

RGB-D datasets using microsoft kinect or similar sensors: a survey (2016, methodology): “Some algorithms tested on this dataset can be found in [4, 56, 71].”
Rendering of Eyes for Eye-Shape Registration and Gaze Estimation (2015, background): “Previous work demonstrates such data to be beneficial for tasks such as body pose estimation [11, 12], object detection/recognition [13, 14, 15, 16], and facial landmark localization [17, 18].”
Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis (2015, methodology): “From each frame, we extracted 68 facial characteristic points (FCPs), such as the position of the left corner of the left eye, etc., using the facial recognition library CLM-Z (Baltrušaitis et al., 2012).”
Select-additive learning: Improving generalization in multimodal sentiment analysis (2016, methodology): “We used the CLM-Z library [30] for extracting facial characteristic points.”
Online modeling for realtime facial animation (2013, methodology): “The method presented in [Baltrušaitis et al. 2012] demonstrates how integrating depth and intensity information in a constrained local model improves tracking performance significantly compared to imagebased tracking alone.”
Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild (2019, methodology): “accuracy in 3D face alignment [45], [46], we used the constrained local neural fields (CLNF) [47] to detect 68 facial landmarks.”
Personalized machine learning for robot perception of affect and engagement in autism therapy (2018, methodology): “This toolkit is based on Conditional Local Neural Fields (CLNF) ( 89 ), a ML model for detection and tracking of 68 ﬁducial facial points, described as 2 dimensional (2D) coordinates ( x, y ) in face images (Fig.”
RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments (2018, background): “CMU Multi-Pie [18] RGB Camera frame 68 Facial landmarks 755,370 ≈300cm X - All BIWI [13] RGB-D Camera frame Head pose vector ≈15,500 100cm X - All ICT 3D Head pose [2] RGB-D Camera frame Head pose vector 14,000 ≈100cm X - All Deep Head Pose [38] RGB-D Camera frame Head pose vector 68,000 ≈200-800cm ”
OpenFace 2.0: Facial Behavior Analysis Toolkit (2018, methodology): “…[70] 13.55 CNN on SynthesEyes + UT [70] 11.12 OpenFace 9.96 UnityEyes [69] 9.95 OpenFace 2.0 9.10 TABLE V: Results comparing our method to previous work for cross dataset gaze estimation on MPIIGaze [79], measure in mean absolute degree error. truth head pose data: BU [16] and ICT-3DHP [10].”
A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances (2022, background): “For example, in a conversation scenario, a person’s emotional state can be demonstrated by the words of speech, the tones of voice, facial expressions and emotion-related body gestures [32].”
Facial Landmark Detection: A Literature Survey (2018, methodology): “For example, In [72][9], 3D face models are used to handle facial expression and pose variations.”
Fake it till you make it: face analysis in the wild using synthetic data alone (2021, methodology): “Building parametric models is challenging, so an alternative is to render 3D scans directly [4, 55, 62, 68].”
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications (2018, methodology): “Local methods often use constraints to stabilize the estimation [SLC11a,CC06,BRM12,AZCP13].”
OpenFace: An open source facial behavior analysis toolkit (2016, methodology): “For comparison, we report the results of using Chehra framework [5], CLM [50], CLM-Z [9], and Regression Forests [21].”
Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild (2013, background): “One of the most promising is the Constrained Local Model (CLM) proposed by Cristinacce and Cootes [5], and various extensions that followed [1, 3, 15, 20].”
SimSensei kiosk: a virtual human interviewer for healthcare decision support (2014, methodology): “The following modules have been integrated in MultiSense: 3D head position-orientation and facial tracking based on GAVAM head tracker [25] and CLM-Z face tracker [3], expression analysis based on SHORE face detector [21] and FACET SDK(3), gaze direction based on OKAO vision(4) and audio analysis ba”
A survey of sentiment analysis in social media (2018, background): “…that 3D morphable models (3DMM), muscle-based models, 3D wire-frame models [27], elastic net model, geometry-based shape models [130], 3D constrained local model (CLM-Z) [10], generalized adaptive view-based appearance model (GAVAM) [76] are important facial expression recognition techniques.”