Learning Deep Features for Discriminative Localization
By Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, Antonio Torralba. 2015
10,756 citations recorded by the source corpus.
In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network (CNN) to have remarkable localization ability despite being trained on imagelevel labels. While this technique was previously proposed as a means for regularizing training, we find that it actually builds a generic localizable deep representation that exposes the implicit attention of CNNs on an image. Despite the apparent simplicity of global average pooling, we are able to achieve 37.1% top-5 error for object localization on ILSVRC 2014 without training on any bounding box annotation. We demonstrate in a variety of experiments that our network is able to localize the discriminative image regions despite just being trained for solving classification task1.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Vladlen Koltun
- Jitendra Malik
- Andrea Vedaldi
- Xiaogang Wang
- Andreas Geiger
- Luc Van Gool
- Abhinav Kumar Gupta
- Marc Pollefeys
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Yi Yang
- Andrew Zisserman
Citations in the corpus
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019, methodology): “Class Activation Map (CAM) (Zhou et al., 2016) for Models with different scaling methods – Our compound scaling method allows the scaled model (last column) to focus on more relevant regions with more object details.”
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “Figure 1: Class activation mapping (CAM) [51] visualizations on ‘Saint Bernard’ and ‘Miniature Poodle’ samples using various augmentation techniques.”
- Unified Perceptual Parsing for Scene Understanding (2018, methodology): “…[32] that although the theoretical receptive field of deep CNN is large enough, the empirical receptive field of deep CNN is relatively much smaller [33], we apply a Pyramid Pooling Module (PPM) from PSPNet [16] on the last layer of the backbone network before feeding it into the top-down branch in…”
- CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, background): “This line of work has found, like us, that simple, supervised classifiers are often effective at detecting manipulations [45, 38].”
- Dynamic Neural Networks: A Survey (2021, methodology): “Although it has been observed that a low resolution might be sufficient for recognizing most ”easy” samples [27], conventional CNNs mostly process all the inputs with the same resolution, inducing considerable redundancy.”
- Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (2016, background): “[34, 48, 11, 32] provide ‘visual explanations’ or spatial maps overlaid on images to highlight the regions that the model focused on while making its predictions.”
- Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning (2017, positive): “Following Zhou et al (2016), we replaced the fully connected layers of our model with convolutions whose activations are spatially averaged to produce class probabilities (i.e. using global average pooling (Lin et al 2014)).”
- Objects that Sound (2017, methodology): “Our approach has similarities with [22] and [36] who used max and average pooling, respectively, to learn object detectors without bounding box annotations in the single visual modality setting, but use ImageNet pretrained networks and image-level labels.”
- Simple Baseline for Visual Question Answering (2015, methodology): “…the answers predicted by the model given the visual feature and question sentence, we first decompose the word contribution of the answer into single words of the question sentence, then we visualize the informative image regions relevant to the answer through the technique proposed in [19].”
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “In particular, [96] included a global average pooling layer between the last convolutional layer of the CNN and the fully-connected layer that predicts the object class.”
- Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, methodology): “Examples are LRP [40], DeepLIFT [41], CAM [42], Grad-CAM [43], Integrated gradients [44], and SmoothGrad [45].”
- SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, methodology): “e image regions used by the model to identify the inputs of the same class. Unfortunately, many of these approaches require modifying and fine-tuning a base model, e.g., Class Activation Mapping (CAM) [49] . Such modifications may alter the behavior of the model, including the malicious behavior that ”
- Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer (2016, methodology): “Our attention maps used for transfer are similar to both gradient-based and activation-based maps mentioned above, which play a role similar to “hints” in FitNets, although we don’t introduce new weights.”
- Spatially Transformed Adversarial Examples (2018, positive): “Zhou et al. (2016a) adopted the spatial transformers for synthesizing novel views of the same object and has shown that a geometric method can produce more realistic results compared to pure pixel-based methods.”
- Recent advances in convolutional neural networks (2015, methodology): “ the stride value with padding, then perform convolution operation on the upsampled input. Recently, deconvolution has been widely used for visualization [8, 35{37], recognition [38{41], localization [42], semantic segmentation [34], visual question answering [43], and super-resolution [44, 45]. 3.1”