Part-Based R-CNNs for Fine-Grained Category Detection

By Ning Zhang, Jeff Donahue, Ross B. Girshick, Trevor Darrell. 2014

1,317 citations recorded by the source corpus.

Semantic part localization can facilitate fine-grained categorization by explicitly isolating subtle appearance differences associated with specific object parts. Methods for pose-normalized representations have been proposed, but generally presume bounding box annotations at test time due to the difficulty of object detection. We propose a model for fine-grained categorization that overcomes these limitations by leveraging deep convolutional features computed on bottom-up region proposals. Our method learns whole-object and part detectors, enforces learned geometric constraints between them, and predicts a fine-grained category from a pose-normalized representation. Experiments on the Caltech-UCSD bird dataset confirm that our method outperforms state-of-the-art fine-grained categorization methods in an end-to-end evaluation without requiring a bounding box at test time.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Fully convolutional networks for semantic segmentation (2014, background): “These include advances in bounding box object detection [4], [5], [6], part and keypoint prediction [7], [8], and local correspondence [8], [9].”
Fine-Grained Image Classification via Combining Vision and Language (2017, background): “Models of ﬁne-grained image classiﬁcation have made great progress in recent years[1, 2, 3, 4, 5], due to the progress of deep neural networks.”
Spatial Transformer Networks (2015, neutral): “In this section we present the results of two further experiments – that of MNIST addition showing spatial transformers acting on multiple objects in Sect.”
Deep Learning Face Attributes in the Wild (2014, background): “Deep learning [18, 34, 23, 7, 19, 32, 31, 13, 33, 22, 3, 28] recently achieved great success in attribute prediction, due to their ability to learn compact and discriminative features.”
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs (2014, background): “, 2014), fine-grained categorization (Zhang et al., 2014), among others.”
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation (2015, methodology): “This transfer learning strategy is widely adopted for object detection [8, 13, 32, 27], semantic segmentation [12, 19, 11, 5, 4, 20], visual tracking [29], and other visual recognition tasks.”
Learning Deep Features for Discriminative Localization (2015, background): “Class activation maps allow us to visualize the predicted class scores on any given image, highlighting the discriminative object parts detected by the CNN.”
On First-Order Meta-Learning Algorithms (2018, methodology): “A classic example of this approach is pretraining using a large dataset (such as ImageNet [2]) and fine-tuning on a smaller dataset (such as a dataset of different species of bird [20]).”
Recent advances in convolutional neural networks (2015, background): “[105] propose a part-based R-CNN which can learn whole-object and part detectors.”
The application of two-level attention models in deep convolutional neural network for fine-grained image classification (2014, methodology): “This starting point is used extensively in recent studies [10, 24], which we adopt as well.”
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition (2015, background): “However, there is a major drawback with many of the current approaches: most of the successes on fine-grained benchmarks are achieved by developing specialized and domain-specific models in order to implicitly or explicitly represent parts, attributes and their relationships [63, 22, 30, 5].”
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition (2020, background): “This issue may also arise with other explainability methods that show or high-light image parts, such as attention mechanisms [10, 13, 44], components [37] and other part-based models e.g. [43, 45, 46].”
This looks like that: deep learning for interpretable image recognition (2018, methodology): “Part R-CNN [53]: 76.4 (bb+anno.)”
Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “Recently variants of such models based on convolutional neural networks (CNNs) [2, 38] have been shown to significantly improve over earlier work that relied on hand-crafted features [1, 11, 39].”
Image Super-Resolution Using Deep Convolutional Networks (2014, background): “First, its structure is intentionally designed with simplicity in mind, and yet provides superior accuracy2 compared with state-of-the-art example-based methods.”