Concept whitening for interpretable image recognition

By Zhi Chen, Yijie Bei, Cynthia Rudin. 2020

377 citations recorded by the source corpus.

What does a neural network encode about a concept as we traverse through the layers? Interpretability in machine learning is undoubtedly important, but the calculations of neural networks are very challenging to understand. Attempts to see inside their hidden layers can be misleading, unusable or rely on the latent space to possess properties that it may not have. Here, rather than attempting to analyse a neural network post hoc, we introduce a mechanism, called concept whitening (CW), to alter a given layer of the network to allow us to better understand the computation leading up to that layer. When a concept whitening module is added to a convolutional neural network, the latent space is whitened (that is, decorrelated and normalized) and the axes of the latent space are aligned with known concepts of interest. By experiment, we show that CW can provide us with a much clearer understanding of how the network gradually learns concepts over layers. CW is an alternative to a batch normalization layer in that it normalizes, and also decorrelates (whitens), the latent space. CW can be used in any layer of the network without hurting predictive performance. There is much interest in ‘explainable’ AI, but most efforts concern post hoc methods. Instead, a neural network can be made inherently interpretable, with an approach that involves making human-understandable concepts (aeroplane, bed, lamp and so on) align along the axes of its latent space.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “For example, Chen et al. (2020) adds constraints to the latent space to decorrelate the concepts and align them with axes in the latent space; this is called “concept whitening.””
Concept Bottleneck Models (2020, background): “Losch et al. (2019) and Chen et al. (2020) also explore learning concept-based models via auxiliary datasets.”
Label-Free Concept Bottleneck Models (2023, background): “Other approaches to make NNs more interpretable include Concept Whitening (Chen et al., 2020) and Concept Embedding Models (Zarlenga et al., 2022).”
Post-hoc Concept Bottleneck Models (2022, background): “Concept Whitening (Chen et al., 2020) aims to align each concept with an individual dimension in a layer.”
From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors (2022, background): “Another opportunity would be utilizing the staggering progress in the explainable deep learning field that has been recently achieved [189, 190].”
Acquisition of chess knowledge in AlphaZero (2021, methodology): “For concept-based explanation, the alignment between the representation and a human’s mental model of the concept remains an open problem [12, 41, 42].”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, background): “Another interesting fine-tuning Concept Whitening [28]approach is that of concept whitening [28], which supplements batchnormalization layers with a linear transformation that learns to align semantic concepts with unit vectors of an activation space. textual or multi-model explanations These provid”
Normalization Techniques in Training DNNs: Methodology, Analysis and Application (2020, background): “There also exist works that impose extra penalties on the loss function to obtain approximately whitened activations [83], [84], [85], [86], [87], or exploit the whitening operation to improve the network’s generalization [88], [89].”