Visualizing Deep Convolutional Neural Networks Using Natural Pre-images

By Aravindh Mahendran, Andrea Vedaldi. 2015

567 citations recorded by the source corpus.

Image representations, from SIFT and bag of visual words to convolutional neural networks (CNNs) are a crucial component of almost all computer vision systems. However, our understanding of them remains limited. In this paper we study several landmark representations, both shallow and deep, by a number of complementary visualization techniques. These visualizations are based on the concept of “natural pre-image”, namely a natural-looking image whose representation has some notable property. We study in particular three such visualizations: inversion, in which the aim is to reconstruct an image from its representation, activation maximization, in which we search for patterns that maximally stimulate a representation component, and caricaturization, in which the visual patterns that a representation detects in an image are exaggerated. We pose these as a regularized energy-minimization framework and demonstrate its generality and effectiveness. In particular, we show that this method can invert representations such as HOG more accurately than recent alternatives while being applicable to CNNs too. Among our findings, we show that several layers in CNNs retain photographically accurate information about the image, with different degrees of geometric and photometric invariance.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, background): “Previous AM research focuses on maximizing the response of pre-trained classifier networks [7, 24, 26, 29].”
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (2016, background): “4 with φ5,4, a loss defined on feature maps of higher level features from deeper network layers with more potential to focus on the content of the images [58, 56, 33].”
Towards the Science of Security and Privacy in Machine Learning (2016, background): “The challenge lies in producing synthetic inputs easily interpreted by humans [114] but faithfully representing the model’s behavior.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization (2016, methodology): “A number of previous works have asserted that deeper representations in a CNN capture higher-level visual constructs [5, 31].”
Stacked Generative Adversarial Networks (2016, background): “One popular approach uses gradientbased optimization to find an image whose representation is close to the one we want to visualize [37].”
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks (2016, background): “Interestingly, previous studies have shown that G trained to invert lower-layer codes (smaller l) results in far better reconstructions than higher-layer codes [25, 6].”
A Survey of Methods for Explaining Black Box Models (2018, background): “Moreover, in Reference [73] this work is extended by comparing various family of methods analyzed for understanding “representations.”
Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models (2017, background): “Unsurprisingly, the development of techniques for “opening” black-box models has recently received a lot of attention in the community [6, 35, 39, 5, 33, 25, 23, 30, 40, 11, 27].”
Interpretable Explanations of Black Boxes by Meaningful Perturbation (2017, neutral): “Most of the existing approaches [19, 16, 8, 7, 9, 19], etc.”
i-RevNet: Deep Invertible Networks (2018, methodology): “The difﬁculty to recover images from their hidden representations is found in many commonly used network architectures (Dosovitskiy & Brox, 2016; Mahendran & Vedaldi, 2016).”
Understanding Image Representations by Measuring Their Equivariance and Equivalence (2014, methodology): “Due to the similarity between HOG and SIFT and due to the fact that HOG can be implemented as a small convolutional neural network (Mahendran and Vedaldi 2016), we focus on the latter in the remainder of the paper.”
Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “/tail lights, wheels, etc. Inverting categories. To understand the properties learned by the B-CNNs we visualize pre-images of a category by “inversion”. We use the framework of Mahendran and Vedaldi [41] to generate an image that produces a high score for a target category C^ based on a B-CNN class”
Understanding intermediate layers using linear classifier probes (2016, background): “An interesting approach is presented in Mahendran and Vedaldi (2015, 2016); Dosovitskiy and Brox (2016) where the authors analyze the set of “equivalent” inputs in the sense that some of the features at a given layer should be preserved.”
Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks (2016, methodology): “There are two general camps within Deep Visualization: activation maximization (Erhan et al., 2009) and code inversion (Mahendran et al., 2015).”
Representation of Linguistic Form and Function in Recurrent Neural Networks (2016, methodology): “…by optimizing random images through backpropagation to maximize the activity of units (Erhan et al., 2009; Simonyan et al., 2014; Yosinski et al., 2015; Nguyen et al., 2016) or to approximate the activation vectors of particular layers (Mahendran and Vedaldi, 2015b; Dosovitskiy and Brox, 2015).”
SoK: Security and Privacy in Machine Learning (2018, background): “The challenge lies in producing human-interpretable synthetic inputs that faithfully represent the model’s behavior [115].”