Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks

By Anh Totti Nguyen, Jason Yosinski, Jeff Clune. 2016

357 citations recorded by the source corpus.

We can better understand deep neural networks by identifying which features each of their neurons have learned to detect. To do so, researchers have created Deep Visualization techniques including activation maximization, which synthetically generates inputs (e.g. images) that maximally activate each neuron. A limitation of current techniques is that they assume each neuron detects only one type of feature, but we know that neurons can be multifaceted, in that they fire in response to many different types of features: for example, a grocery store class neuron must activate either for rows of produce or for a storefront. Previous activation maximization techniques constructed images without regard for the multiple different facets of a neuron, creating inappropriate mixes of colors, parts of objects, scales, orientations, etc. Here, we introduce an algorithm that explicitly uncovers the multiple facets of each neuron by producing a synthetic visualization of each of the types of images that activate a neuron. We also introduce regularization methods that produce state-of-the-art results in terms of the interpretability of images obtained by activation maximization. By separately synthesizing each type of image a neuron fires in response to, the visualizations have more appropriate colors and coherent global structure. Multifaceted feature visualization thus provides a clearer and more comprehensive description of the role of each neuron.

Subfields: Biophysics, Materials Chemistry, Media Technology, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Materials Chemistry

In Media Technology

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Stacked Generative Adversarial Networks (2016, background): “However, in the above formulation, the generator might choose to ignore the highlevel conditional information hi+1, and generate plausible lower-level representations ĥi from scratch.”
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks (2016, methodology): “Instead of previous works, which directly optimized an image so that it highly activates a neuron h in Φ and optionally satisfies hand-designed priors embedded in the cost function [5, 7, 9, 6], here we optimize in the input code of an image generator network G such that G outputs an image that high”
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, methodology): “Earlier research is similar, “ synthesizing preferred input ” using deep generator networks [53].”
Generative Counterfactual Introspection for Explainable Deep Learning (2019, methodology): “To understand how components of the network work, a variety of the methods have been introduced to visualization the feature (or pattern) the given neuron or layer aim to capture [16]–[18] or examining the representation of the high-level concept in the latent representations [19].”
Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models (2017, methodology): “This includes the development of methods which help to better understand what the model has learned (i.e., it’s representation) [12, 24, 29] as well as techniques for explaining individual predictions [19, 35, 39, 5, 26].”
Methods for interpreting and understanding deep neural networks (2017, background): “[53], who demonstrate in the context of image classification, the benefit of interpreting a class ωc using multiple local prototypes instead of a single global one.”
Improving Interpretability of Deep Neural Networks with Semantic Information (2017, methodology): “Recently, many research efforts have been devoted to interpreting hidden features of DNNs [12, 25, 38, 37], and have made several steps towards interpretability, e.g ., the de-convolutional networks [37] to visualize the layers of convolutional networks, and the activation maximization [12] to…”
Towards Better Analysis of Deep Convolutional Neural Networks (2016, methodology): “Existing methods can be classified into two categories, namely, code inversion [12, 35, 58] and activation maximization [13, 41, 45, 53, 57].”
Interpretability of deep learning models: A survey of results (2017, background): “[12] showed that CNNs learn the global structure, details, and context of objects rather than a small number of local discriminating features.”
Identifying Individual Facial Expressions by Deconstructing a Neural Network (2016, methodology): “Explanation methods such as [3,26,22,12,13,7,27,25,19] have been able to successfully solve this important open problem.”
An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents (2018, methodology): “Finally, there are other analysis tools that could be implemented, which might illuminate other interesting properties of DRL algorithms and learned representation, e.g. the image perturbation analysis of Greydanus et al. [13] or a variety of sophisticated neuron visualization techniques [33, 32].”
A Powerful Generative Model Using Random Weights for the Deep Image Representation (2016, methodology): “The VGG-based random weight network created as described in the method section is used as our reference network, denoted as ranVGG in the following discussion.”
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers (2018, neutral): “Some studies generated hundreds of these non-deterministic synthetic images and clustered them to see how variations in the class activation maximization algorithm affects the output image [79].”