PANDA: Pose Aligned Networks for Deep Attribute Modeling

By Ning Zhang, Manohar Paluri, Marc'Aurelio Ranzato, Trevor Darrell, Lubomir D. Bourdev. 2013

546 citations recorded by the source corpus.

We propose a method for inferring human attributes (such as gender, hair style, clothes style, expression, action) from images of people under large variation of viewpoint, pose, appearance, articulation and occlusion. Convolutional Neural Nets (CNN) have been shown to perform very well on large scale object recognition problems. In the context of attribute classification, however, the signal is often subtle and it may cover only a small part of the image, while the image is dominated by the effects of pose and viewpoint. Discounting for pose variation would require training on very large labeled datasets which are not presently available. Part-based models, such as poselets [4] and DPM [12] have been shown to perform well for this problem but they are limited by shallow low-level features. We propose a new method which combines part-based models and deep learning by training pose-normalized CNNs. We show substantial improvement vs. state-of-the-art methods on challenging attribute classification tasks in unconstrained settings. Experiments confirm that our method outperforms both the best part-based methods on this problem and conventional CNNs trained on the full bounding box of the person.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Caffe: Convolutional Architecture for Fast Feature Embedding (2014, background): “…for the operation of the network as a whole: a forward pass that takes the inputs and produces the outputs, and a backward pass that takes the gradient with respect to the output, and computes the gradients with respect to the parameters and to the inputs, which are in turn…”
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition (2014, methodology): “In processing test images, our method is 24-102× faster than the R-CNN method, while achieving better or comparable accuracy on Pascal VOC 2007.”
Deformable part models are convolutional neural networks (2014, background): “As a result, HOG-based detectors are still used in a wide range of systems, such as recent hybrid deep/conventional approaches [3, 46], and especially where region-based methods are ill-suited (poselets [1] being a prime example).”
Deep Learning Face Attributes in the Wild (2014, background): “Fig.13 demonstrates that our method performs well when dataset size is small, but the performance of PANDA-l drops significantly.”
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition (2014, methodology): “Finally, we have to point out that, given enough computational resources, optimizing the CNN features for specific tasks/datasets would probably boost the performance of the simplistic system even further [29, 15, 51, 43, 41].”
Part-Based R-CNNs for Fine-Grained Category Detection (2014, methodology): “PANDA [49] learned part-speciﬁc deep convolutional networks whose location was conditioned on HOG-based poselet models.”
Towards Deep Neural Network Architectures Robust to Adversarial Examples (2014, background): “Deep neural networks have recently led to signiﬁcant improvement in countless areas of machine learning, from speech recognition to computer vision Krizhevsky et al. (2012); Dahl et al. (2012); Taigman et al. (2013); Zhang et al. (2013).”
C3D: Generic Features for Video Analysis (2014, background): “These features are normally the activations of the network’s last few fully con-1 nected layers which are showed to be useful for transfer learning tasks [22, 54, 56].”
Contextual Action Recognition with R*CNN (2015, methodology): “PANDA [34] uses CNNs trained for each poselet type.”
Structural-RNN: Deep Learning on Spatio-Temporal Graphs (2015, background): “They have been applied to multiple problems including image segmentation, pose estimation, document processing [64, 6, 38, 3] etc.”
DeepEdge: A multi-scale bifurcated deep network for top-down contour detection (2014, methodology): “eatures have been optimized for the task of object class recognition, they have been shown to be highly effective for other image-analysis tasks, including object detection [11], attribute prediction [29], and image style recognition [14]. The network was trained on 1:2 million images and it include”
Beyond frontal faces: Improving Person Recognition using multiple cues (2015, methodology): “We are inspired by the work of Zhang et al. [38], which uses deep convolutional networks trained on poselet detected patches for attribute classiﬁcation.”
Part Detector Discovery in Deep Convolutional Neural Networks (2014, neutral): “Zhang et al. [18] use poselets for part discovery as well as detection and calculate features for each region using a deep CNN.”
Dense Optical Flow Prediction from a Static Image (2015, background): “Current work on CNNs have largely focused on recognition tasks both in images and video [12, 9, 4, 23, 29, 20, 22].”
Multi-Task CNN Model for Attribute Prediction (2015, methodology): “Because this work is mainly related to the topics of Semantic attributes, Multi-task learning and Deep CNN, we briefly review the most recent literature on these approaches including the following.”
Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification (2016, methodology): “State-of-the-art methods for person attribute prediction are based on deep convolutional neural networks [43, 24, 5, 48].”