Object-Part Attention Model for Fine-Grained Image Classification

By Yuxin Peng, Xiangteng He, Junjie Zhao. 2017

379 citations recorded by the source corpus.

Fine-grained image classification is to recognize hundreds of subcategories belonging to the same basic-level category, such as 200 subcategories belonging to the bird, which is highly challenging due to large variance in the same subcategory and small variance among different subcategories. Existing methods generally first locate the objects or parts and then discriminate which subcategory the image belongs to. However, they mainly have two limitations: 1) relying on object or part annotations which are heavily labor consuming; and 2) ignoring the spatial relationships between the object and its parts as well as among these parts, both of which are significantly helpful for finding discriminative parts. Therefore, this paper proposes the object-part attention model (OPAM) for weakly supervised fine-grained image classification and the main novelties are: 1) object-part attention model integrates two level attentions: object-level attention localizes objects of images, and part-level attention selects discriminative parts of object. Both are jointly employed to learn multi-view and multi-scale features to enhance their mutual promotion; and 2) Object-part spatial constraint model combines two spatial constraints: object spatial constraint ensures selected parts highly representative and part spatial constraint eliminates redundancy and enhances discrimination of selected parts. Both are jointly employed to exploit the subtle and local differences for distinguishing the subcategories. Importantly, neither object nor part annotations are used in our proposed approach, which avoids the heavy labor consumption of labeling. Compared with more than ten state-of-the-art methods on four widely-used datasets, our OPAM approach achieves the best performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Yu Qiao
3. Dacheng Tao
4. Tao Mei
5. Jiashi Feng
6. Luc Van Gool
7. Mohit Bansal
8. Trevor Darrell

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Ming-Ming Cheng
3. Dacheng Tao
4. Ming-Hsuan Yang
5. Jiashi Feng
6. Yu Qiao
7. Xiaogang Wang
8. Yi Yang

Citations in the corpus

• Do Better ImageNet Models Transfer Better? (2018, methodology): “Several papers have proposed methods to make better use of CNN features and thus improve the efficacy of transfer learning [43, 11, 42, 23, 73, 63, 13, 41, 54].”
• Relational Knowledge Distillation (2019, methodology): “For such datasets, effective adaptation to specific characteristics of the domain may be crucial; recent methods for fine-grained classification focus on localizing discriminative parts of target-domain objects [23, 44, 48].”
• WildFish: A Large Benchmark for Fish Recognition in the Wild (2018, methodology): “Strategy 69.0 RACNN [14] 65.4 OPAM [29] 71.6 Im2Recipe [31] 71.6 all the categories.”
• Emergent Correspondence from Image Diffusion (2023, background): “Establishing visual correspondences between different images is crucial for various computer vision tasks such as Structure-from-Motion / 3D reconstruction [2, 74, 61, 75], object tracking [23, 93], image recognition [64, 81, 9] and segmentation [51, 48, 73, 30].”
• Learning Rich Part Hierarchies With Progressive Attention Networks for Fine-Grained Image Recognition (2020, neutral): “• OPAM [53]: localizing object and discriminative parts based on their spacial constraint without object or part annotations.”
• GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks (2021, background): “Plenty of tasks suffer from this problem, like medical diagnosis [13, 24] or fine-grained image classification [29, 36].”
• Fine-Grained Image Analysis With Deep Learning: A Survey (2021, neutral): “0% – OPAM [85] IEEE TIP 2018 VGG-16 Not given 85.”
• Delving Deep Into Label Smoothing (2020, neutral): “The fine-grained image classifi-cation task [60], [61], [62], [63], [64] focuses on distinguishing [18] 102 2040 6149 Cars [20] 196 8144 8041 Aircrafts [21] 90 6667 3333 subordinate categories within entry-level categories [27], [65], [66], [67].”
• Attribute-Aware Attention Model for Fine-grained Representation Learning (2018, background): “OPAM [29] proposes object-part attention model to localize object and discriminative parts.”
• Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images (2020, background): “Generally, these fine-grained image learning methods follow a two-stage strategy that first locates key objects in the images and then discriminates their belonging subclasses based on the identified objects [22].”
• GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (2018, neutral): “8∗ [29] Food-101 75,750 25,250 101 93.”
• Attention mechanisms in computer vision: A survey (2021, background): “, image classification [5, 6], object detection [7, 8], semantic segmentation [9, 10], face recognition [11, 12], person re-identification [13, 14], action recognition [15, 16], few-shot learning [17, 18], medical image processing [19, 20], image generation [21, 22], pose estimation [23], super reso”
• Destruction and Construction Learning for Fine-Grained Image Recognition (2019, methodology): “[21] proposed a part spatial constraint to make sure that the model could select discriminative regions, and a specialized clustering algorithm is used to integrate the features of these regions.”