Face recognition in unconstrained videos with matched background similarity

By Lior Wolf, Tal Hassner, Itay Maoz. 2011

1,471 citations recorded by the source corpus.

Recognizing faces in unconstrained videos is a task of mounting importance. While obviously related to face recognition in still images, it has its own unique characteristics and algorithmic requirements. Over the years several methods have been suggested for this problem, and a few benchmark data sets have been assembled to facilitate its study. However, there is a sizable gap between the actual application needs and the current state of the art. In this paper we make the following contributions. (a) We present a comprehensive database of labeled videos of faces in challenging, uncontrolled conditions (i.e., ‘in the wild’), the ‘YouTube Faces’ database, along with benchmark, pair-matching tests1. (b) We employ our benchmark to survey and compare the performance of a large variety of existing video face recognition techniques. Finally, (c) we describe a novel set-to-set similarity measure, the Matched Background Similarity (MBGS). This similarity is shown to considerably improve performance on the benchmark tests.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

FaceNet: A unified embedding for face recognition and clustering (2015, background): “Youtube Faces DB [21] is a new dataset that has gained popularity in the face recognition community [17, 15].”
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks (2018, methodology): “…fc6 256 / / ReLU fc7 128 / / ReLU fc8 Attack Methodology: the attack is implemented on images from the YouTube Aligned Face dataset [45]. layer ﬁlter stride padding activation conv1 96x3x11x11 4 0 / conv3 conv4 conv4 conv5 CleanDigit0 conv3 384x256x3x3 1 1 ReLU conv4 384x384x3x3 1…”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, methodology): “We use the YouTube Aligned Face dataset for evaluation, which is a pre-processed dataset of images taken from the YouTube Faces dataset [67].”
Towards a Neural Statistician (2016, methodology): “We use the Youtube Faces Database from Wolf et al. [32] sampling frames randomly without replacement from each video, we use a set size of 5 frames.”
How Image Degradations Affect Deep CNN-Based Face Recognition? (2016, background): “With the recent advances in deep convolutional neural networks, researchers have reached encouraging improvements in face recognition accuracy on the LFW [Hu07] and YouTube Faces [WHM11] datasets [Su15, PVZ15, SKP15].”
SphereFace: Deep Hypersphere Embedding for Face Recognition (2017, neutral): “LFW dataset [9] includes 13,233 face images from 5749 different identities, and YTF dataset [35] includes 3,424 videos from 1,595 different individuals.”
VGGFace2: A Dataset for Recognising Faces across Pose and Age (2017, background): “3/530 - - 2007 YTF [23] 1, 595 3, 425 videos - - 2011 CelebFaces+ [20] 10, 177 202, 599 19.”
Joint Unsupervised Learning of Deep Representations and Image Clusters (2016, neutral): “02, CMU-PIE [48], Youtube-Face (YTF)) [57].”
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale (2015, background): “Finally, Best-Rowden et al. [3] performed an interesting Mechanical Turk study to evaluate human recognition rates on LFW and YouTube Faces datasets.”
Learning Face Representation from Scratch (2014, methodology): “YTF [28] is another dataset based on the name list of LFW but it’s created for video based face recognition.”
MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition (2016, methodology): “Available people images IJB-A [17] public 500 5712 LFW [7,8] public 5K 13K YFD [14] public 1595 3425 videos CelebFaces [15] public 10K 202K CASIA-WebFace [16] As shown in Table 1, our training dataset is considerably larger than the publicly available datasets.”
CosFace: Large Margin Cosine Loss for Deep Face Recognition (2018, methodology): “Accuracy (%) of CosFace with different margin parameters m on LFW[13] and YTF [43].”
Deeply learned face representations are sparse, selective, and robust (2014, background): “2% on YouTubeFaces [35] for face veriﬁcation (vs. 91 .”
Age and Gender Estimation of Unfiltered Faces (2014, background): “and the YouTube faces [2] collections, similar data sets are”
Deep Imbalanced Learning for Face Recognition and Attribute Prediction (2018, background): “We test on three popular large-scale datasets: LFW [1], YTF [64] and MegaFace [2].”
Range Loss for Deep Face Recognition with Long-Tailed Training Data (2016, methodology): “In this section, we evaluate our range loss based models on two well known face recognition benchmarks, LFW and YTF data sets.”
Two-stage Image Classification Supervised by a Single Teacher Single Student Model (2019, methodology): “(c) MUCT, (d) FEI, (e) YouTubeFace.”
Discriminative Deep Metric Learning for Face Verification in the Wild (2014, methodology): “In our experiments, we follow the standard evaluation protocol [34] and test our method for unconstrained face verification with 5000 video pairs.”