Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs

By Robin Rombach, Patrick Esser, Björn Ommer. 2020

19 citations recorded by the source corpus.

To tackle increasingly complex tasks, it has become an essential ability of neural networks to learn abstract representations. These task-specific representations and, particularly, the invariances they capture turn neural networks into black box models that lack interpretability. To open such a black box, it is, therefore, crucial to uncover the different semantic concepts a model has learned as well as those that it has learned to be invariant to. We present an approach based on INNs that (i) recovers the task-specific, learned invariances by disentangling the remaining factor of variation in the data and that (ii) invertibly transforms these recovered invariances combined with the model representation into an equally expressive one with accessible semantic concepts. As a consequence, neural network representations become understandable by providing the means to (i) expose their semantic meaning, (ii) semantically modify a representation, and (iii) visualize individual learned semantic concepts and invariances. Our invertible approach significantly extends the abilities to understand black box models by enabling post-hoc interpretations of state-of-the-art networks without compromising their performance. Our implementation is available at this https URL .

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Taming Transformers for High-Resolution Image Synthesis (2020, methodology): “[13, 48] demonstrate similar gains when using an unconditional normalizing flow for the second stage, and [40] when using a conditional normalizing flow.”
High Fidelity Visualization of What Your Self-Supervised Representation Knows About (2021, methodology): “Since our model is able to project any representation to the manifold of real images, we follow the same experiment protocol as Rombach et al. (2020) to visualize how adversarial examples are seen by RCDM.”
Stochastic Image-to-Video Synthesis using cINNs (2021, methodology): “Each block consists of actnorm [15], affine coupling layers [4], and fixed shuffling layers, following previous work [24].”
Understanding invariance via feedforward inversion of discriminatively trained classifiers (2021, methodology): “Learning based methods use a separate training set of {logits, image pixels} pairs to learn a decoder network that maps a logit vector to an image (Dosovitskiy and Brox, 2016a;b; Nash et al., 2019; Rombach et al., 2020).”