Visual Concept Recognition and Localization via Iterative Introspection

By Amir Rosenfeld, Shimon Ullman. 2016

23 citations recorded by the source corpus.

Convolutional neural networks have been shown to develop internal representations, which correspond closely to semantically meaningful objects and parts, although trained solely on class labels. Class Activation Mapping (CAM) is a recent method that makes it possible to easily highlight the image regions contributing to a network’s classification decision. We build upon these two developments to enable a network to re-examine informative image regions, which we term introspection. We propose a weakly-supervised iterative scheme, which shifts its center of attention to increasingly discriminative regions as it progresses, by alternating stages of classification and introspection. We evaluate our method and show its effectiveness over a range of several datasets, where we obtain competitive or state-of-the-art results: on Stanford-40 Actions, we set a new state-of the art of 81.74%. On FGVC-Aircraft and the Stanford Dogs dataset, we show consistent improvements over baselines, some of which include significantly more supervision.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks (2018, methodology): “[19] proposed an iterative method to crop the relevant regions of the image for fine-grained classification.”
Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition (2018, background): “For instance, the work in [17] extends [34] and establishes a cascaded multi-stage framework, which reﬁnes the attention region by iteration.”