Inverting Supervised Representations with Autoregressive Neural Density Models

By Charlie Nash, Nate Kushman, Christopher K. I. Williams. 2018

26 citations recorded by the source corpus.

We present a method for feature interpretation that makes use of recent advances in autoregressive density estimation models to invert model representations. We train generative inversion models to express a distribution over input features conditioned on intermediate model representations. Insights into the invariances learned by supervised models can be gained by viewing samples from these inversion models. In addition, we can use these inversion models to estimate the mutual information between a model's inputs and its intermediate representations, thus quantifying the amount of information preserved by the network at different stages. Using this method we examine the types of information preserved at different layers of convolutional neural networks, and explore the invariances induced by different architectural choices. Finally we show that the mutual information between inputs and network layers decreases over the course of training, supporting recent work by Shwartz-Ziv and Tishby (2017) on the information bottleneck theory of deep learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

High Fidelity Visualization of What Your Self-Supervised Representation Knows About (2021, neutral): “On the other hand, Nash et al. (2019) proposed to use an autoregressive model, in particular PixelCNN++ (Salimans et al., 2017), to specifically study the invariances that each layer of a DN inherits.”
Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment (2019, background): “Much research has been working on interpreting and understanding neural networks [28], [65], [29], [66], [67], [68].”
Understanding invariance via feedforward inversion of discriminatively trained classifiers (2021, methodology): “Learning based methods use a separate training set of {logits, image pixels} pairs to learn a decoder network that maps a logit vector to an image (Dosovitskiy and Brox, 2016a;b; Nash et al., 2019; Rombach et al., 2020).”
Network-to-Network Translation with Conditional Invertible Neural Networks (2020, methodology): “To address this, [48] learns an autoregressive model conditioned on speciﬁc layers of pre-trained models and [63] a GAN based decoder conditioned on a feature pyramid of a pre-trained classiﬁer.”
Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs (2020, methodology): “[44] used samples from an autoregressive model of images conditioned on a feature representation to gain insights into the representation’s invariances.”
Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment (2019, background): “In general, these inversion approaches fall into two categories: optimization-based inversion [21, 30, 37, 39, 43, 44, 72] and, most similar to our approach, training-based inversion [10, 17, 18, 53].”