Bird Species Categorization Using Pose Normalized Deep Convolutional Nets

By Steve Branson, Grant Van Horn, Serge J. Belongie, Pietro Perona. 2014

552 citations recorded by the source corpus.

We propose an architecture for fine-grained visual categorization that approaches expert human performance in the classification of bird species. Our architecture first computes an estimate of the object's pose; this is used to compute local image features which are, in turn, used for classification. The features are computed by applying deep convolutional nets to image patches that are located and normalized by the pose. We perform an empirical study of a number of pose normalization schemes, including an investigation of higher order geometric warping functions. We propose a novel graph-based clustering algorithm for learning a compact pose normalization space. We perform a detailed investigation of state-of-the-art deep convolutional feature implementations and fine-tuning feature learning for fine-grained classification. We observe that a model that integrates lower-level feature layers with pose-normalized extraction routines and higher-level feature layers with unaligned image features works best. Our experiments advance state-of-the-art performance on bird species recognition, with a large improvement of correct classification rates over previous methods (75% vs. 55-65%).

Subfields: Computer Vision and Pattern Recognition, Developmental Biology, Ecology, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Developmental Biology

In Ecology

In Molecular Biology

Citations in the corpus

Fine-Grained Image Classification via Combining Vision and Language (2017, methodology): “Object and Part annotations are directly used in training phase to learn a strongly supervised deformable part-based model [16] or directly used to fine-tune the pretrained CNN model [17].”
Spatial Transformer Networks (2015, neutral): “While previous work such as [2] explicitly define parts of the bird, training separate detectors for these parts with supplied keypoint training data, the ST-CNN is able to discover and learn part detectors in a data-driven manner without any additional supervision.”
The application of two-level attention models in deep convolutional neural network for fine-grained image classification (2014, methodology): “To further improve the performance, part level annotation is also used in training phase to learn strongly-supervised deformable part-based model [1, 27] or directly used to finetune pre-trained CNN [4].”
This looks like that: deep learning for interpretable image recognition (2018, methodology): “, [50, 49, 3, 24, 15]); other attention models generate attention with “blackbox” methods – e.”
Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “Recently variants of such models based on convolutional neural networks (CNNs) [2, 38] have been shown to significantly improve over earlier work that relied on hand-crafted features [1, 11, 39].”
Part Detector Discovery in Deep Convolutional Neural Networks (2014, neutral): “1 After submission, two additional publications [30,31] were published reporting 73.”
This Looks Like That: Deep Learning for Interpretable Image Recognition (2018, methodology): “It relates closely to Branson et al. (2014) [4], who used a greedy facility location algorithm to identify a set of prototypes for use in bird species identification.”
A survey on deep learning-based fine-grained object classification and semantic segmentation (2017, methodology): “Part-based R-CNN [19] AlexNet BBox + Parts BBox 76.4 Part-based R-CNN [19] AlexNet BBox + Parts − 73.9 Multi-proposal consensus [22] AlexNet BBox BBox 80.3 PoseNorm [24] Alexnet BBox + Parts − 75.7 PS-CNN [26] AlexNet BBox + Parts BBox 76.2 Deep LAC [28] AlexNet BBox BBox 80.3”
Picking Deep Filter Responses for Fine-Grained Image Recognition (2016, methodology): “Most works choose the output of a CNN as feature representation directly [2], [15], [27], [32].”
Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding (2018, methodology): “Deep Localization, Alignment and Classi˙cation (DeepLAC) [24], Semantic Part Detection and Abstraction (SPDA-CNN) [46], Part-RCNN [47], Part Alignment-based (PA-CNN) [20], Pose Normalized CNN (PNCNN) [1], Picking Deep Filter Responses (PDFR) [48], Multiple Granularity (MG-CNN) [40], Spatial Transfor”
Audio Visual Attribute Discovery for Fine-Grained Object Recognition (2018, methodology): “…methods into two groups based on their distinct supervised label, which are the visual fine-grained annotations, e.g. the visual part label, (Branson et al. 2014; Huang et al. 2016; Krause et al. 2015; Liu et al. 2012; Xie et al. 2013; Zhang et al. 2016a; 2014; 2016b) and the text…”
Mask-CNN: Localizing parts and selecting descriptors for fine-grained bird species categorization (2018, methodology): “9% Pose Normalized CNNs [30] X X Alex-Net×3 13,512 75.”
Coarse-to-Fine Description for Fine-Grained Visual Categorization (2016, background): “Given a test image, Branson et al. [3] first infer the keypoints and then employ these keypoints to align the generated local parts.”
Deep LAC: Deep localization, alignment and classification for fine-grained recognition (2015, background): “Branson et al. [4] studied higher-order geometric warping to align parts.”
Evaluation of deep convolutional nets for document image classification and retrieval (2015, background): “Additionally, in problems where region-specific information is important, it is potentially better to encode this information in multiple networks trained on the regions of interest than in a single network trained on the entire image [4].”
Dynamic Computational Time for Visual Attention (2017, background): “The second group attempts to localize discriminative parts to effectively deal with large intra-class variation as well as subtle inter-class variation [55, 56, 57, 47, 9].”