Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition

By Ming Sun, Yuchen Yuan, Feng Zhou, Errui Ding. European Conference on Computer Vision, 2018

393 citations recorded by the source corpus.

Attention-based learning for fine-grained image recognition remains a challenging task, where most of the existing methods treat each object part in isolation, while neglecting the correlations among them. In addition, the multi-stage or multi-scale mechanisms involved make the existing methods less efficient and hard to be trained end-to-end. In this paper, we propose a novel attention-based convolutional neural network (CNN) which regulates multiple object parts among different input images. Our method first learns multiple attention region features of each input image through the one-squeeze multi-excitation (OSME) module, and then apply the multi-attention multi-class constraint (MAMC) in a metric learning framework. For each anchor feature, the MAMC functions by pulling same-attention same-class features closer, while pushing different-attention or different-class features away. Our method can be easily trained end-to-end, and is highly efficient which requires only one training stage. Moreover, we introduce Dogs-in-the-Wild, a comprehensive dog species dataset that surpasses similar existing datasets by category coverage, data volume and annotation quality. Extensive experiments are conducted to show the substantial improvements of our method on four benchmark datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ab9e438195989056f471086f9becdfb3b2f90459

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Attentive Pairwise Interaction for Fine-Grained Classification (2020, methodology): “2018a), multi-attention multi-class constraint (Sun et al. 2018), etc.”
Destruction and Construction Learning for Fine-Grained Image Recognition (2019, methodology): “[20] proposed a onesqueeze multi-excitation module to learns multiple attention region features of each input image, and then apply a multi-attention multi-class constraint in a metric learning framework.”
Zero-Shot Learning from scratch (ZFS): leveraging local compositional representations (2020, background): “Attention over local features is commonly used in image captioning (Li et al., 2017), visual question answering (Kim et al., 2018) and ﬁne-grained classiﬁcation (Sun et al., 2018).”
Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification (2021, neutral): “Note that although some of previous methods like [57] propose to equip the backbone network with spatial attention modules, here we follow the mainstreams [47, 31, 65, 50, 19, 28, 28, 25, 62, 64] that learn basic feature maps and attentions separately.”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “[27] incorporated channel attentions [129] and metric learning [130] to enforce the correlations among different attended regions.”
Measuring Dataset Granularity (2019, background): “Examples of such modules include non-linear feature interactions [32, 15, 7] and attention mechanisms [22, 53, 14, 56, 46, 34, 55].”
See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification (2019, background): “…ResNet-101 [7] 83.5 Inception-V3 [26] 83.7 B-CNN [16] 84.1 ST-CNN [9] 84.1 PDFR [37] 84.5 RA-CNN [5] 85.4 GP-256 [31] 85.8 MA-CNN [38] 86.5 MAMC [25] 86.5 PC [4] 86.9 DFL-CNN [29] 87.4 NTS-Net [33] 87.5 MPN-COV [12] 88.7 WS-DAN 89.4 [23] 80.5 ResNet-101 [7] 87.2 Inception-V3 [26] 87.4 B-CNN…”
Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-Grained Image Recognition (2019, methodology): “• MAMC [27]: Multi-attention multi-class constraint network, which learns multiple attentions by conducting multi-class constraint over attended features.”
Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization (2020, neutral): “8 OSME+MAMC [22] 86.”
Locality and compositionality in zero-shot learning (2019, background): “Attention over local features is commonly used in image captioning (Li et al., 2017), visual question answering (Kim et al., 2018) and ﬁne-grained classiﬁcation (Sun et al., 2018).”