Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition

By Heliang Zheng, Jianlong Fu, Tao Mei, Jiebo Luo. 2017

932 citations recorded by the source corpus.

Recognizing fine-grained categories (e.g., bird species) highly relies on discriminative part localization and part-based fine-grained feature learning. Existing approaches predominantly solve these challenges independently, while neglecting the fact that part localization (e.g., head of a bird) and fine-grained feature learning (e.g., head shape) are mutually correlated. In this paper, we propose a novel part learning approach by a multi-attention convolutional neural network (MA-CNN), where part generation and feature learning can reinforce each other. MA-CNN consists of convolution, channel grouping and part classification sub-networks. The channel grouping network takes as input feature channels from convolutional layers, and generates multiple parts by clustering, weighting and pooling from spatially-correlated channels. The part classification network further classifies an image by each individual part, through which more discriminative fine-grained features can be learned. Two losses are proposed to guide the multi-task learning of channel grouping and part classification, which encourages MA-CNN to generate more discriminative parts from feature channels and learn better fine-grained features from parts in a mutual reinforced way. MA-CNN does not need bounding box/part annotation and can be trained end-to-end. We incorporate the learned parts from MA-CNN with part-CNN for recognition, and show the best performances on three challenging published fine-grained datasets, e.g., CUB-Birds, FGVC-Aircraft and Stanford-Cars.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Environmental Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Model Architectures

Citations in the corpus

CrossTransformers: spatially-aware few-shot transfer (2020, background): “Part-based correspondences for recognition [103] have been particularly successful in fine-grained face retrieval and recognition [48, 95].”
Dynamic Neural Networks: A Survey (2021, neutral): “Sa & Sp Image segmentation [34], [129], [146], [148], [150], [154], [156], [216], [217], [218], [219], [220], Image-to-image translation [221], Object detection [110], [111], [147], [148], [164], Semantic image synthesis [222], [223], [224], Computer Image denoising [225], Fine-grained classificatio”
Neural Prototype Trees for Interpretable Fine-grained Image Recognition (2020, methodology): “part-based methods [56, 61], saliency maps [6, 13, 60] or representer points [53].”
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition (2020, background): “This issue may also arise with other explainability methods that show or highlight part of an image, such as attention mechanisms [10, 15, 47], components [38] and other part-based models e.”
Towards Interpretable Reinforcement Learning Using Attention Augmented Agents (2019, background): “image classification and captioning [12, 13, 14, 15, 16, 17, 18, 19, 20], text classification [21, 22], generative models [6, 23, 24], object tracking [25], and reinforcement learning [26, 27, 28].”
This looks like that: deep learning for interpretable image recognition (2018, methodology): “, RA-CNN [7] uses another neural network (attention proposal network) to decide where to look next; multi-attention CNN [52] uses aggregated convolutional feature maps as “part attentions.”
Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning (2018, methodology): “To capture subtle visual differences, visual attention [60, 16, 69] and deep metric learning [45, 10] are often used.”
Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization (2020, methodology): “5 MA-CNN [9] 86.”
Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective (2022, background): “With the backbone network pre-trained on ImageNet frozen and unfrozen, two kinds of transferring experiments are conducted on the CUB-200-2011 (CUB200) [39] and the FGVC-Aircraft (Aircraft) [27] datasets.”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “[83], [84] were the first to incorporate attention process-”
Batch Coherence-Driven Network for Part-Aware Person Re-Identification (2020, background): “Existing works have demonstrated that specific channels in the top layers of CNNs describe specific visual patterns [18], [43], [49]–[51] (i.”
Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks (2018, background): “More recently, [17] expanded this idea to multiple attention locations in the image, instead of a single one.”
Deep Learning on Small Datasets without Pre-Training using Cosine Loss (2019, background): “In such scenarios, pre-training on similar tasks with large amounts of data such as the ImageNet dataset [9] has become the de facto standard [51, 12], for example in the domain of fine-grained recognition [23, 56, 8, 37].”
Destruction and Construction Learning for Fine-Grained Image Recognition (2019, methodology): “[42] adopted a channel grouping network to generate multiple parts by clustering, then classi-”