Concept Bottleneck Models

By Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen O. Mussmann, Emma Pierson, Been Kim, Percy Liang. 2020

1,388 citations recorded by the source corpus.

We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis? State-of-the-art models today do not typically support the manipulation of concepts like “the existence of bone spurs”, as they are trained end-to-end to go directly from raw input (e.g., pixels) to output (e.g., arthritis severity). We revisit the classic idea of first predicting concepts that are provided at training time, and then using these concepts to predict the label. By construction, we can intervene on these concept bottleneck models by editing their predicted concept values and propagating these changes to the final prediction. On x-ray grading and bird identification, concept bottleneck models achieve competitive accuracy with standard end-to-end models, while enabling interpretation in terms of high-level clinical concepts (“bone spurs”) or bird attributes (“wing color”). These models also allow for richer human-model interaction: accuracy improves significantly if we can correct model mistakes on concepts at test time.

Subfields: Artificial Intelligence, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Safety, Robustness & Interpretability

Citations in the corpus

SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, neutral): “, 2019) and interpretable classes (Koh et al., 2020; kuan Yeh et al., 2020).”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, background): “Recent work in this area aims to disentangle the latent space with respect to a collection of predefined concepts (Chen et al., 2020; Koh et al., 2020; Losch et al., 2019; Adel et al., 2018).”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…et al., 2018), ImageNet-C (Hendrycks and Dietterich, 2019), and similar ImageNet variants (Geirhos et al., 2018); and datasets that crop out objects and replace their backgrounds, as in the Backgrounds Challenge (Xiao et al., 2020) and other similar datasets (Sagawa et al., 2020; Koh et al., 2020).”
Label-Free Concept Bottleneck Models (2023, methodology): “One approach to make deep learning more interpretable is through Concept Bottleneck Models (CBMs) (Koh et al., 2020).”
General Multi-label Image Classification with Transformers (2020, methodology): “We also consider two baselines from [23].”
Acquisition of chess knowledge in AlphaZero (2021, methodology): “Concept-based interpretability Concept-based interpretability methods build on the idea of network probing [19], and use human understandable concepts to explain neural network decisions in terms that users can understand [7, 8, 9, 10, 11].”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, methodology): “Koh et al. [41] learn models that ﬁrst predict human labeled concepts, then use concept scores to predict model.”
Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification (2022, methodology): “Our method builds on Concept Bottleneck Models (CBM) [25], which construct predictors through a linear combination of human-designed concepts.”
Post-hoc Concept Bottleneck Models (2022, methodology): “Particularly, we try the joint training strategy from Koh et al. (Koh et al., 2020), where it was reported to 0 .”
Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application (2020, neutral): “It bears some similarity to concept bottleneck models (Koh et al., 2020), though distinct theoretical motivation.”
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, neutral): “…learn features that describe ‘prototypical concepts’ or ‘prototypes’ present in individual inputs to the model, including ProtoPNet [147], ProtoTree [148], ProtoPShare [149], Concept Bottleneck Models [150], Concept Activation Vectors [146], Concept Embedding Models [151] or Concept Atlases [152].”
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task (2022, background): “Finally, latent saliency maps can be viewed as a generalization of the TCAV approach Kim et al. (2018); Ghorbani et al. (2019); Koh et al. (2020).”
Low-cost concept-based localized explanations: How far can we get with training-free approaches? (2026, unknown): “Fine-grained labeling by concept-level naming Existing concept-extraction techniques, such as TCAV [7], ACE [9], and Concept Bottleneck Models [11], rely on curated concept datasets and supervised training pipelines.”
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation (2025, unknown): “…modifications to attention mechanisms (Correia et al., 2019), applying Bayesian standard Gamma priors (Duan et al., 2024), incorporating discrete sparse concept layers (Koh et al., 2020), and promoting sparse activations in large language models (Mirzadeh et al., 2023; Zhang et al., 2024).”
Dataset Featurization: Uncovering Natural Language Features through Unsupervised Data Reconstruction (2025, unknown): “A parallel line of work explores Concept Bottleneck Models, which achieve interpretability by learning models over interpretable intermediate features (Koh et al., 2020).”