Synthesizing the preferred inputs for neurons in neural networks via deep generator networks

By Anh Totti Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, Jeff Clune. 2016

739 citations recorded by the source corpus.

Deep neural networks (DNNs) have demonstrated state-of-the-art results on many pattern recognition tasks, especially vision classification problems. Understanding the inner workings of such computational brains is both fascinating basic science that is interesting in its own right - similar to why we study the human brain - and will enable researchers to further improve DNNs. One path to understanding how a neural network functions internally is to study what each of its neurons has learned to detect. One such method is called activation maximization (AM), which synthesizes an input (e.g. an image) that highly activates a neuron. Here we dramatically improve the qualitative state of the art of activation maximization by harnessing a powerful, learned prior: a deep generator network (DGN). The algorithm (1) generates qualitatively state-of-the-art synthetic images that look almost real, (2) reveals the features learned by each neuron in an interpretable way, (3) generalizes well to new datasets and somewhat well to different network architectures without requiring the prior to be relearned, and (4) can be considered as a high-quality generative method (in this case, by generating novel, creative, interesting, recognizable images).

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “Furthermore, by coupling the scene parsing network with the recent image synthesis technique proposed in [24], we could also synthesize a scene image given the semantic label mask.”
CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “…is that the produced images often leave the space of natural images (Nguyen et al., 2015), or fool the system through adversarial means (Goodfellow et al., 2014b), thus a body of work aims to discover ‘natural image priors’ to constrain which images may be produced (Nguyen et al., 2016; 2017).”
Unrolled Generative Adversarial Networks (2016, background): “(14) This concept of backpropagating to generate images has been widely used in visualizing features from discriminative networks (Simonyan et al., 2013; Yosinski et al., 2015; Nguyen et al., 2016) and has been applied to explore the visual manifold of GANs in (Zhu et al.”
GAN Inversion: A Survey (2021, background): “[72] optimize over input codes of a generator network that was trained to reconstruct images from hidden layers.”
Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, neutral): “using GAN) to capture the prior [70, 73, 74].”
Scene Parsing through ADE20K Dataset (2017, methodology): “Furthermore, by coupling the scene parsing network with the recent image synthesis technique proposed in [23], we could also synthesize a scene image given the semantic label mask.”
Photographic Image Synthesis with Cascaded Refinement Networks (2017, methodology): “[11], also referred to as a perceptual loss or feature matching [3, 7, 17, 19, 27, 28].”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “In the same direction, the authors in [268] introduced a Deep Generator Network (DGN) that generates the most representative image for a given output neuron in a CNN.”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “Visualizations can be created by optimizing an input image using gradient descent [39], by sampling images that maximize activation [49], or by training a generative network to create such images [50].”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “[82] proposed an approach based on activation maximization—synthesizing the preferred inputs for neurons in neural networks—via a learned prior in the form of a deep generator network to produce a global interpretable model for image recognition.”
A Survey of Methods for Explaining Black Box Models (2018, methodology): “The method f proposed in Reference [80] by Nguyen et al.”
What do we need to build explainable AI systems for the medical domain? (2017, background): “…G can be trained to minimize log(1 − D ( G ( z ))); in other words, D and G play the following two-player minimax game with value function V ( G, D ): (5) Nguyen et al. (2016) proposed to build a prototype for ω c by incorporating such a generative model in the activation maximization framework.”
Methods for interpreting and understanding deep neural networks (2017, methodology): “[51] proposed to build a prototype for ωc by incorporating such generative model in the activation maximization framework.”
Learning how to explain neural networks: PatternNet and PatternAttribution (2017, background): “…decisions and to gain insight into how these models operate, a variety techniques have been proposed (Simonyan et al., 2014; Yosinski et al., 2015; Nguyen et al., 2016; Baehrens et al., 2010; Bach et al., 2015; Montavon et al., 2017; Zeiler & Fergus, 2014; Springenberg et al., 2015; Zintgraf et…”