The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results

By Jie Shen, Stefanos P. Zafeiriou, Grigorios G. Chrysos, Jean Kossaifi, Georgios Tzimiropoulos, Maja Pantic. 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), 2015

332 citations recorded by the source corpus.

Detection and tracking of faces in image sequences is among the most well studied problems in the intersection of statistical machine learning and computer vision. Often, tracking and detection methodologies use a rigid representation to describe the facial region 1, hence they can neither capture nor exploit the non-rigid facial deformations, which are crucial for countless of applications (e.g., facial expression analysis, facial motion capture, high-performance face recognition etc.). Usually, the non-rigid deformations are captured by locating and tracking the position of a set of fiducial facial landmarks (e.g., eyes, nose, mouth etc.). Recently, we witnessed a burst of research in automatic facial landmark localisation in static imagery. This is partly attributed to the availability of large amount of annotated data, many of which have been provided by the first facial landmark localisation challenge (also known as 300-W challenge). Even though now well established benchmarks exist for facial landmark localisation in static imagery, to the best of our knowledge, there is no established benchmark for assessing the performance of facial landmark tracking methodologies, containing an adequate number of annotated face videos. In conjunction with ICCV'2015 we run the first competition/challenge on facial landmark tracking in long-term videos. In this paper, we present the first benchmark for long-term facial landmark tracking, containing currently over 110 annotated videos, and we summarise the results of the competition.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/07d95be4922670ef2f8b11997e0c00eb643f3fca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Novel Space-Time Representation on the Positive Semidefinite Cone for Facial Expression Recognition (2017, background): “In particular, advances in landmarks detection [3, 34, 44] have opened the door to accurate geometry-driven approaches.”
InverseFaceNet: Deep Monocular Inverse Face Rendering (2017, methodology): “Qualitative results on LFW [22] and 300-VW [53].”
Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors (2018, unknown): “300-VW [30, 34, 6].”
How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) (2017, methodology): “Traditionally, the metric used for face alignment is the point-to-point Euclidean distance normalized by the interocular distance [10, 29, 33].”
InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image (2017, methodology): “We then show qualitative and quantitative results and comparisons on the datasets CelebA [29], LFW (Labeled Faces in the Wild) [16], FaceWarehouse [7], Thomas [13], Volker [47] and 300-VW (300 Videos in the Wild) [40].”
OpenFace 2.0: Facial Behavior Analysis Toolkit (2018, methodology): “Datasets The facial landmark detection capability was evaluated on two publicly available datasets: IJB-FL [36], and 300VW [60] test set.”
Towards Fast, Accurate and Stable 3D Dense Face Alignment (2020, background): “Specifically, Menpo-3D provides 3D facial landmarks for 55 videos from 300-VW [43] competition.”
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild (2020, unknown): “We show some challenging results predicted by RetinaFace-ResNet50 on 300VW [4], AFW [7], Pascal [3] and WIDER FACE [6].”
Facial Landmark Detection: A Literature Survey (2018, background): “There are additional detailed references [16] [78] [86] that provide original evaluations by running the software and implementations of known algorithms on diﬀerent databases.”
Automatic Analysis of Facial Actions: A Survey (2019, background): “The 300 Videos in the Wild [163] is currently the bestestablished benchmark on this topic.”
TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement (2023, methodology): “viewed from the front or back); thus most methods rely on large supervised datasets [2, 9, 44, 50, 67, 72], although some works track surface meshes [17, 62].”
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction (2017, background): “[41, 42] (coarse network without refinement) on 300-VW [8, 48, 56] (left) and LFW [19] (right).”
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz (2017, methodology): “We combined four different datasets: CelebA [42], LFW [29], FaceWarehouse [17], and 300-VW [19, 55, 63].”
Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection (2019, background): “These keypoints include the corners of a mouth, the boundary of eyes, the tip of a nose, etc [36, 35, 21].”