Recover Canonical-View Faces in the Wild with Deep Neural Networks

By Zhenyao Zhu, Ping Luo, Xiaogang Wang, Xiaoou Tang. 2014

109 citations recorded by the source corpus.

Face images in the wild undergo large intra-personal variations, such as poses, illuminations, occlusions, and low resolutions, which cause great challenges to face-related applications. This paper addresses this challenge by proposing a new deep learning framework that can recover the canonical view of face images. It dramatically reduces the intra-person variances, while maintaining the inter-person discriminativeness. Unlike the existing face reconstruction methods that were either evaluated in controlled 2D environment or employed 3D information, our approach directly learns the transformation from the face images with a complex set of variations to their canonical views. At the training stage, to avoid the costly process of labeling canonical-view images from the training set by hand, we have devised a new measurement to automatically select or synthesize a canonical-view image for each identity. As an application, this face recovery approach is used for face verification. Facial features are learned from the recovered canonical-view face images by using a facial component-based convolutional neural network. Our approach achieves the state-of-the-art performance on the LFW dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

FaceNet: A unified embedding for face recognition and clustering (2015, methodology): “The works of [15, 17, 23] all employ a complex system of multiple stages, that combines the output of a deep convolutional network with PCA for dimensionality reduction and an SVM for classiﬁcation.”
Naive-Deep Face Recognition: Touching the Limit of LFW Benchmark or Not? (2015, background): “Throughout the history of LFW benchmark, surprising improvements are obtained with recent deep learning techniques [17, 14, 13, 10, 12].”
Surpassing Human-Level Face Verification Performance on LFW with GaussianFace (2014, methodology): “More recently, Zhu et al. [63] learned the transformation from face images under various poses and lighting conditions to a canonical view with a deep convolutional network.”
Face Model Compression by Distilling Knowledge from Neurons (2016, background): “As the emergence of big training data, Deep Neural Networks (DNNs) recently attained great breakthroughs in face recognition [23, 20, 21, 22, 19, 15, 29, 30, 28] and become applicable in many commercial platforms such as social networks, e-commerce, and search engines.”
Facial Landmark Detection by Deep Multi-task Learning (2014, background): “1 Introduction Facial landmark detection is a fundamental component in many face analysis tasks, such as facial attribute inference [17], face verification [15,22,23,35], and face recognition [33,34].”
Effective face frontalization in unconstrained images (2014, methodology): “Finally, [41] described a deep-learning based method for estimating canonical views of faces.”
DeepID3: Face Recognition with Very Deep Neural Networks (2015, background): “Using deep neural networks to learn effective feature representations has become popular in face recognition [12, 20, 17, 22, 14, 13, 18, 21, 19, 15].”
A Comprehensive Survey on Pose-Invariant Face Recognition (2015, methodology): “The performance of the reviewed approaches is evaluated [Zhu et al. 2013; Kan et al. 2014] Multiview subspace learning Linear models CCA [Li et al. 2009], PLS [Sharma and Jacobs 2011], GMLDA [Sharma et al. 2012b], Tied Factor Analysis [Prince et al. 2008] Nonlinear models Kernel CCA [Akaho 2006], De”
An incremental face recognition system based on deep learning (2017, background): “[6] employ a deep network to align faces into a canonical frontal view and then learn a CNN that classifies each face as belonging to a known identity.”
Deep Face Recognition: A Survey (2018, methodology): “[66], [67] selected canonicalview images according to the face images’ symmetry and”
Learning Deep Representation for Face Alignment with Auxiliary Attributes (2014, background): “or of 9.15% on the challenging IBUG subset). 1 Introduction Facial landmark detection is a fundamental component in many face analysis tasks, such as facial attribute inference [20], face verication [18,30,31,44], and face recognition [42,43]. Though great strides have been made in this eld [8,9,10,”
Deep Learning Face Representation from Predicting 10,000 Classes (2014, methodology): “Recently, deep models such as ConvNets [24] have been proved effective for extracting high-level visual features [11, 20, 14] and are used for face verification [18, 5, 31, 32, 36].”
Labeled Faces in the Wild: A Survey (2016, background): “Zhu et al, 2014.”