Interpretability Beyond Classification Output: Semantic Bottleneck Networks

By Max Maria Losch, Mario Fritz, Bernt Schiele. 2019

75 citations recorded by the source corpus.

Today's deep learning systems deliver high performance based on end-to-end training. While they deliver strong performance, these systems are hard to interpret. To address this issue, we propose Semantic Bottleneck Networks (SBN): deep networks with semantically interpretable intermediate layers that all downstream results are based on. As a consequence, the analysis on what the final prediction is based on is transparent to the engineer and failure cases and modes can be analyzed and avoided by high-level reasoning. We present a case study on street scene segmentation to demonstrate the feasibility and power of SBN. In particular, we start from a well performing classic deep network which we adapt to house a SB-Layer containing task related semantic concepts (such as object-parts and materials). Importantly, we can recover state of the art performance despite a drastic dimensionality reduction from 1000s (non-semantic feature) to 10s (semantic concept) channels. Additionally we show how the activations of the SB-Layer can be used for both the interpretation of failure cases of the network as well as for confidence prediction of the resulting output. For the first time, e.g., we show interpretable segmentation results for most predictions at over 99% accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

In Safety, Robustness & Interpretability

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “Koh et al. (2020), Losch et al. (2019) and Adel et al. (2018) try to directly learn a concept classifier in the latent space.”
Concept Bottleneck Models (2020, background): “Losch et al. (2019) and Chen et al. (2020) also explore learning concept-based models via auxiliary datasets.”
Label-Free Concept Bottleneck Models (2023, methodology): “CBM : Most related to our approach are Concept Bottleneck Models (Koh et al., 2020; Losch et al., 2019) which create a layer before the last fully connected layer where each neuron corresponds to a human interpretable concept.”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, methodology): “These approaches can be seen as a post-hoc version of the Semantic Bottleneck [87] architecture, only not replacing the complete later part of the model, but just learning connections from the bottleneck to the succeeding trained layer.”