Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors

By Xuanyi Dong, Shoou-I Yu, Xinshuo Weng, Shih-En Wei, Yi Yang, Yaser Sheikh. 2018

206 citations recorded by the source corpus.

In this paper, we present supervision-by-registration, an unsupervised approach to improve the precision of facial landmark detectors on both images and video. Our key observation is that the detections of the same landmark in adjacent frames should be coherent with registration, i.e., optical flow. Interestingly, coherency of optical flow is a source of supervision that does not require manual labeling, and can be leveraged during detector training. For example, we can enforce in the training loss function that a detected landmark at framet-1 followed by optical flow tracking from framet-1 to framet should coincide with the location of the detection at framet. Essentially, supervision-by-registration augments the training loss function with a registration loss, thus training the detector to have output that is not only close to the annotations in labeled images, but also consistent with registration on large amounts of unlabeled videos. End-to-end training with the registration loss is made possible by a differentiable Lucas-Kanade operation, which computes optical flow registration in the forward pass, and back-propagates gradients that encourage temporal coherency in the detector. The output of our method is a more precise image-based facial landmark detector, which can be applied to single images or video. With supervision-by-registration, we demonstrate (1) improvements in facial landmark detection on both images (300W, ALFW) and video (300VW, Youtube-Celebrities), and (2) significant reduction of jittering in video detections.

Subfields: Computer Vision and Pattern Recognition, Dermatology, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Dermatology

In Neurology

Citations in the corpus

Searching for a Robust Neural Architecture in Four GPU Hours (2019, background): “Designing an efficient and effective neural architecture requires substantial human effort and takes a long time [7, 9, 10, 12, 14, 20, 37, 44].”
Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “17 - CPM + SBR [35] CPM 2 .”
Generalizing a Person Retrieval Model Hetero- and Homogeneously (2018, background): “Beneﬁt from the remarkable success of deep learning [24,25,26,27,28], recent works [2,29,30] attempt to predict pseudo-labels to unlabeled samples based on the deep learning framework.”
3FabRec: Fast Few-Shot Face Alignment by Reconstruction (2019, unknown): “21 SBR [16] 2.”
Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection (2019, methodology): “Following [20, 11], we report the results for the 49 inner points on the category C subset of the 300-VW test set, which has 26338 frames.”
Multi-Stream Deep Neural Networks for RGB-D Egocentric Action Recognition (2019, methodology): “[88] presented an unsupervised supervision-by-registration approach for facial landmark de-”
Style Aggregated Network for Facial Landmark Detection (2018, background): “Some researchers extend the landmark detection in the image to video settings [22, 13, 40] or 3D settings [6, 47].”
A Bottom-Up Clustering Approach to Unsupervised Person Re-Identification (2019, methodology): “In recent years, some transfer learning methods (Hehe et al. 2018; Peng et al. 2016; Deng et al. 2018) are proposed upon the success of deep learning (Dong et al. 2018a; Li, Tang, and Mei 2018).”
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics (2019, background): “ntroduction Multi-object tracking (MOT) is an essential component technology for many vision applications such as autonomous driving [1,2,3], robot collision prediction [4,5] and video face alignment [6, 7,8]. Due to the signiﬁcant advance in object detection [9,10,11,12,13,14,15,16,17], there has b”