A Semi-automatic Methodology for Facial Landmark Annotation

By Christos Sagonas, Georgios Tzimiropoulos, Stefanos P. Zafeiriou, Maja Pantic. 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013

422 citations recorded by the source corpus.

Developing powerful deformable face models requires massive, annotated face databases on which techniques can be trained, validated and tested. Manual annotation of each facial image in terms of landmarks requires a trained expert and the workload is usually enormous. Fatigue is one of the reasons that in some cases annotations are inaccurate. This is why, the majority of existing facial databases provide annotations for a relatively small subset of the training images. Furthermore, there is hardly any correspondence between the annotated land-marks across different databases. These problems make cross-database experiments almost infeasible. To overcome these difficulties, we propose a semi-automatic annotation methodology for annotating massive face datasets. This is the first attempt to create a tool suitable for annotating massive facial databases. We employed our tool for creating annotations for MultiPIE, XM2VTS, AR, and FRGC Ver. 2 databases. The annotations will be made publicly available from http://ibug.doc.ic.ac.uk/ resources/facial-point-annotations/. Finally, we present experiments which verify the accuracy of produced annotations.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/23ec870360a5b8bde88e3b2ca1aef9ff22fd718b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

AgeDB: The First Manually Collected, In-the-Wild Age Database (2017, methodology): “Therefore, their N = 68 facial landmarks points are localized using the face detector from [1], trained with images provided from 300-W challenge [44, 41, 43] and subsequently employed to compute a mean reference shape.”
Learn to Combine Multiple Hypotheses for Accurate Face Alignment (2013, methodology): “We strictly follow the protocol, and only use the provided training annotations generated by [11, 25, 15, 17, 18] to train our model.”
DEEP eye contact detector: Robust eye contact bid detection using convolutional neural network (2017, methodology): “Facial images and landmarks are provided from LFPW [15], HELEN [10], AFW [27] and IBUG [17, 18, 19], and we categorized them as eye contacted (upper) or averted (lower) samples.”
Face Alignment at 3000 FPS via Regressing Local Binary Features (2014, background): “300-W (68 landmarks) is short for 300 Faces in-theWild [23].”
How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) (2017, methodology): “Such methods are now considered to have largely “solved” the 2D face alignment problem for faces with controlled pose variation like the ones of LFPW [2], Helen [22] and 300-W [30].”
Incremental Face Alignment in the Wild (2014, methodology): “4, we use both the LFPW [3, 26, 25] and Helen [17, 26, 25] datasets.”
Facial Landmark Detection: A Literature Survey (2018, methodology): “In [80], an iterative semi-automatic landmark annotation method is proposed.”
Modeling Facial Geometry Using Compositional VAEs (2018, methodology): “We initialize the process by using the 2D landmarks provided by [31], to compute the head pose and general expression, and then solve the MAP of Eq.”
Learning Residual Images for Face Attribute Manipulation (2016, methodology): “We trained the detection model on the 300-W dataset [24].”
(2013) 300 Faces in-the-Wild Challenge: the first facial landmark localization challenge. In: 2013 IEEE International (2017, methodology): “W was created by using the semi-automatic methodology for facial landmark annotation proposed as in [15, 16] followed by additional manual correction.”
Mnemonic Descent Method: A Recurrent Process Applied for End-to-End Face Alignment (2016, background): “In recent years, the problem of face alignment has seen substantial improvement, partially due to the introduction of large datasets of unconstrained (in-thewild ) images [6] [29] [65], which have been consistently re-annotated [39] [40] [38].”
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression (2017, methodology): “The dataset has been produced by fitting a 3DMM built from the combination of the Basel [17] and FaceWarehouse [3] models to the unconstrained images of the 300W dataset [22] using the multi-feature fitting approach of [20], careful initialisation and by constraining the solution using a sparse set ”
Face Alignment in Full Pose Range: A 3D Total Solution (2015, background): “With 300 W, we adopt the proposed face profiling to generate 61,225 samples across large poses (1,786 from IBUG, 5,207 from AFW, 16,556 from LFPW and 37,676 from HELEN, XM2VTS is not used), which is further flipped to 122,450 samples.”
Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild (2013, methodology): “In order to evaluate the ability of CLNF to generalise on unseen datasets we evaluated our approach on the datasets labelled for the 300 Faces in-the-Wild Challenge [14].”
Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment (2014, neutral): “3) are provided by [24].”
Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks (2017, methodology): “The face images involved in 300W have been semi-automatically annotated by 68 facial landmarks [50].”
Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors (2018, background): “ process, the detector can remember certain personalized details in an unsupervised fashion to achieve more precise and stable facial landmark detection. 4. Evaluation and Results 4.1. Datasets 300-W [26,28,29] provides annotations for 3837 face images with 68 landmarks. We follow [40,19,41] to spli”