Beyond frontal faces: Improving Person Recognition using multiple cues

By Ning Zhang, Manohar Paluri, Yaniv Taigman, Rob Fergus, Lubomir D. Bourdev. 2015

165 citations recorded by the source corpus.

We explore the task of recognizing peoples' identities in photo albums in an unconstrained setting. To facilitate this, we introduce the new People In Photo Albums (PIPA) dataset, consisting of over 60000 instances of ~2000 individuals collected from public Flickr photo albums. With only about half of the person images containing a frontal face, the recognition task is very challenging due to the large variations in pose, clothing, camera viewpoint, image resolution and illumination. We propose the Pose Invariant PErson Recognition (PIPER) method, which accumulates the cues of poselet-level person recognizers trained by deep convolutional networks to discount for the pose variations, combined with a face recognizer and a global recognizer. Experiments on three different settings confirm that in our unconstrained setup PIPER significantly improves on the performance of DeepFace, which is one of the best face recognizers as measured on the LFW dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

ALICE: Active Learning with Contrastive Natural Language Explanations (2020, methodology): “Dataset We also evaluate ALICE on the People in Photo Album Relation dataset (Zhang et al., 2015; Sun et al., 2017).”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “The collaborative model for PIPA is a VGG-style [54] 10-layer CNN with two convolution blocks consisting of one convolutional layer and max pooling, followed by three convolution blocks consisting of two convolutional layers and max pooling, followed by two fully connected layers.”
Defeating Image Obfuscation with Deep Learning (2016, background): “…by humans, and real-world protections deployed by popular systems such as YouTube); and 4) Oh et al. only evaluate a single dataset (the People in Photo Albums [55]), whereas we evaluate our attack against diverse datasets, including MNIST, CIFAR-10, AT&T Database of Faces, and FaceScrub.”
A Hybrid Model for Identity Obfuscation by Face Replacement (2018, background): “The PIPA dataset [39] is the largest social media dataset (37,107 Flickr images with 2,356 annotated individuals), which shows people in diverse events, activities and poses.”
Faceless Person Recognition: Privacy Implications in Social Media (2016, methodology): “The PIPA dataset defines train, validation, and test partitions (∼17k, ∼ 5k, ∼ 8k photos respectively), each containing disjoint sets of identities [29].”
I-Pic: A Platform for Privacy-Compliant Image Capture (2016, methodology): “Due to this generalization, it outperforms other cutting-edge face recognition systems in a social media photo setting [13], where individuals often do not pose for the camera.”
A Domain Based Approach to Social Relation Recognition (2017, methodology): “We extend the dataset called People in Photo Album (PIPA) [36] by 26,915 person pair annotations.”
Person Recognition in Personal Photo Collections (2015, methodology): “We build our paper upon the recently introduced PIPA dataset [40] which is two orders of magnitude larger (~40k images, ~2k identities), more diverse, and also provides identity annotations when the face is not visible.”
How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis (2020, methodology): “As databases have grown larger to meet the demands of increasingly complex computer vision tasks, databases are now being built by scraping publicly available image data, such as the People In Photo Albums (PIPA) database, which used Flickr images [175]).”
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning (2018, methodology): “To validate the assumption, we now present an experiment to quantify user variations on two public image datasets (PIPA [86] and OpenImages [49]).”
Blind Backdoors in Deep Learning Models (2020, methodology): “To train a model for counting the number of faces in an image, we use the PIPA dataset [100] with photos of 2,356 individuals.”
A Survey of Privacy Attacks in Machine Learning (2020, background): “Such an active attacker reached an attack accuracy of 92.1%.”
Person Re-identification in the Wild (2016, methodology): “However, the setting in [42] is not typical for person re-ID where pedestrians are observed by surveillance cameras and faces are not clear enough.”
Natural and Effective Obfuscation by Head Inpainting (2017, methodology): “We use the PIPA dataset [34], the largest social media dataset to date with people in diverse events, activities, and poses.”
Overlearning Reveals Sensitive Attributes (2019, background): “PIPA is a set of over 60,000 photos of 2,000 individuals gathered from public Flickr photo al-bums (Piper project page; Zhang et al., 2015).”
Person Recognition in Personal Photo Collections (2017, methodology): “Our h + b method is the minimal system matching Zhang et al.’s PIPER [1] ( 83 .”
Adversarial Image Perturbation for Privacy Protection A Game Theory Perspective (2017, methodology): “We use the val1 subset of PIPA, consisting of 4820 instances of 366 identities (val-original split1 in [27] terminology) as the test set.”
Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild (2019, methodology): “We further compare PAM to existing alternatives on the IJB-A and PIPA [19] benchmarks.”
Understanding and Controlling User Linkability in Decentralized Learning (2018, methodology): “We use two public, academic image datasets, PIPA [82] and OpenImages [37] to train this classifier.”
MovieNet: A Holistic Dataset for Movie Understanding (2020, methodology): “Then we use ResNet50 pretrained on PIPA [84] to extract the body feature and face feature as representation.”