Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild

By Iacopo Masi, Feng-Ju Chang, Jongmoo Choi, Shai Harel, Jungyeon Kim, Kanggeon Kim, Jatuporn Toy Leksut, Stephen Rawls, Yue Wu, Tal Hassner and 5 more. 2019

102 citations recorded by the source corpus.

We propose a method designed to push the frontiers of unconstrained face recognition in the wild with an emphasis on extreme out-of-plane pose variations. Existing methods either expect a single model to learn pose invariance by training on massive amounts of data or else normalize images by aligning faces to a single frontal pose. Contrary to these, our method is designed to explicitly tackle pose variations. Our proposed Pose-Aware Models (PAM) process a face image using several pose-specific, deep convolutional neural networks (CNN). 3D rendering is used to synthesize multiple face poses from input images to both train these models and to provide additional robustness to pose variations at test time. Our paper presents an extensive analysis of the IARPA Janus Benchmark A (IJB-A), evaluating the effects that landmark detection accuracy, CNN layer selection, and pose model selection all have on the performance of the recognition pipeline. It further provides comparative evaluations on IJB-A and the PIPA dataset. These tests show that our approach outperforms existing methods, even surprisingly matching the accuracy of methods that were specifically fine-tuned to the target dataset. Parts of this work previously appeared in [1] and [2] .

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Adversarial Invariance (2018, background): “The most popular approach to expand the data size and prevent overﬁtting in deep learning has been synthetic data augmentation [3, 5, 9–11], where multiple copies of data samples are created by altering variations of certain known nuisance factors.”
Transferability and Hardness of Supervised Classification Tasks (2019, background): “A set of face images, for instance, can have multiple labels for each image, representing tasks such as recognition [39, 40] and classification of various attributes [34].”
Recurrent Convolutional Strategies for Face Manipulation Detection in Videos (2019, background): “Since it has been shown in [29] that face alignment is beneficial for face recognition, we employ face alignment here as well.”
FSGAN: Subject Agnostic Face Swapping and Reenactment (2019, methodology): “Instead, they used a fixed 3D face shape as the proxy [13, 28, 29].”