Shortcut learning in deep neural networks
By Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard S. Zemel, Wieland Brendel, Matthias Bethge, Felix Wichmann. 2020
3,155 citations recorded by the source corpus.
Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today’s machine intelligence. Numerous success stories have rapidly spread all over science, industry and society, but its limitations have only recently come into focus. In this Perspective we seek to distil how many of deep learning’s failures can be seen as different symptoms of the same underlying problem: shortcut learning. Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios. Related issues are known in comparative psychology, education and linguistics, suggesting that shortcut learning may be a common characteristic of learning systems, biological and artificial alike. Based on these observations, we develop a set of recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications. Deep learning has resulted in impressive achievements, but under what circumstances does it fail, and why? The authors propose that its failures are a consequence of shortcut learning, a common characteristic across biological and artificial systems in which strategies that appear to have solved a problem fail unexpectedly under different circumstances.
Subfields: Artificial Intelligence, Scaling & Learning Theory, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yoshua Bengio
- Quoc V. Le
- Percy Liang
- Jianfeng Gao
- Luke S. Zettlemoyer
- Yejin Choi
- Kyunghyun Cho
In Scaling & Learning Theory
- Yoshua Bengio
- Aaron C. Courville
- Pascal Vincent
- Ilya Sutskever
- Andrew Gordon Wilson
- Sepp Hochreiter
- Taiji Suzuki
- Yann LeCun
In Safety, Robustness & Interpretability
- Nicholas Carlini
- Dan Hendrycks
- Matthias Hein
- Dawn Song
- Adina Williams
- Jörn-Henrik Jacobsen
- Alan L. Yuille
- Wieland Brendel
Citations in the corpus
- PaLM: Scaling Language Modeling with Pathways (2022, background): “In all cases, the correct prediction can be unambiguously inferred (by a human) from the provided context, so the gap between the stereotypical and gotcha subsets is a strong measure of how much a model is relying on statistical shortcuts (Geirhos et al., 2020).”
- Learning Transferable Visual Models From Natural Language Supervision (2021, background): “What explains this discrepancy? Various ideas have been suggested and studied (Ilyas et al., 2019; Geirhos et al., 2020).”
- Measuring Massive Multitask Language Understanding (2020, background): “This has the benefit of obviating concerns about spurious training set annotation artifacts (Geirhos et al., 2020; Hendrycks et al., 2019b) and is in stark contrast to the previous paradigm of identically distributed training and test sets.”
- Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “However, recent work has shown that NLP systems built with this pattern often rely on “shortcuts” (Geirhos et al., 2020), which may be based on spurious phenomena in the training data (McCoy et al.”
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, background): “While the research community must create robust models that generalize to new scenarios, the robustness literature [6, 9] lacks consensus on evaluation benchmarks and contains many dissonant hypotheses.”
- CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP (2021, methodology): “Our contributions are: (a) we propose CLOOB, a new contrastive learning method, (b) we propose modern Hopfield networks to reinforce covariance structures, (c) we propose InfoLOOB as an objective to avoid saturation as observed with InfoNCE, and provide theoretical justifications for optimizing…”
- Robust Speech Recognition via Large-Scale Weak Supervision (2022, background): “A model that achieves “superhuman” performance when trained on a dataset can still make many basic errors when evaluated on another, possibly precisely because it is exploiting those dataset-specific quirks that humans are oblivious to (Geirhos et al., 2020).”
- Factual Probing Is [MASK]: Learning vs. Learning to Recall (2021, methodology): “Our study also relates to a larger category of work diagnosing “shortcut learning” (Geirhos et al., 2020) in neural NLP models.”
- Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, background): “Geirhos et al. [2020] illustrate how deep neural networks rely on shortcuts , or decision rules that do not extrapolate well to out-of-distribution data and are often based on incidental associations, for performance.”
- WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “Despite their ubiquity in real-world deployments, these types of distribution shifts are under-represented in the datasets widely used in the ML community today (Geirhos et al., 2020).”
- Inductive biases for deep learning of higher-level cognition (2020, background): “Instead, current systems are generally not robust to changes in distribution (Peters et al., 2017b; Geirhos et al., 2020; Hendrycks et al., 2021; Koh et al., 2021; Schneider et al., 2020), adversarial examples (Goodfellow et al.”
- What Do Models Learn from Question Answering Datasets? (2020, background): “There is also growing interest in probing what models learn from datasets (McCoy et al., 2019; Geirhos et al., 2020; Richardson and Sabharwal, 2020; Si et al., 2020).”
- Distilling Model Failures as Directions in Latent Space (2022, neutral): “More broadly, the training dataset might contain spurious correlations, encouraging the model to depend on prediction rules that do not generalize to deployment (Xiao et al., 2020; Geirhos et al., 2020; DeGrave et al., 2021).”
- Faith and Fate: Limits of Transformers on Compositionality (2023, background): “[25] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann.”
- Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “Note that analyses on the adversarial examples [58, 77] show the connections to the understanding of the deep learning process and”
- Perturbation Augmentation for Fairer NLP (2022, background): “Models that utilize demographic terms as lexical “shortcuts” (Geirhos et al., 2020) during classification will have a larger change in their predictions than models that do not, with the latter being deemed “more fair” by our metric.”
- End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “However, methods trained with multiple frames can become overly reliant on this shortcut [241] and suffer from catastrophic failure during deployment.”
- Advances, challenges and opportunities in creating data for trustworthy AI (2022, background): “There is also growing recognition that state-of-the-art AI models often pick up spurious correlations and biases in the development data 13 .”