Shortcut learning in deep neural networks

By Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard S. Zemel, Wieland Brendel, Matthias Bethge, Felix Wichmann. 2020

3,155 citations recorded by the source corpus.

Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today’s machine intelligence. Numerous success stories have rapidly spread all over science, industry and society, but its limitations have only recently come into focus. In this Perspective we seek to distil how many of deep learning’s failures can be seen as different symptoms of the same underlying problem: shortcut learning. Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios. Related issues are known in comparative psychology, education and linguistics, suggesting that shortcut learning may be a common characteristic of learning systems, biological and artificial alike. Based on these observations, we develop a set of recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications. Deep learning has resulted in impressive achievements, but under what circumstances does it fail, and why? The authors propose that its failures are a consequence of shortcut learning, a common characteristic across biological and artificial systems in which strategies that appear to have solved a problem fail unexpectedly under different circumstances.

Subfields: Artificial Intelligence, Scaling & Learning Theory, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Yoshua Bengio
  3. Quoc V. Le
  4. Percy Liang
  5. Jianfeng Gao
  6. Luke S. Zettlemoyer
  7. Yejin Choi
  8. Kyunghyun Cho

In Scaling & Learning Theory

  1. Yoshua Bengio
  2. Aaron C. Courville
  3. Pascal Vincent
  4. Ilya Sutskever
  5. Andrew Gordon Wilson
  6. Sepp Hochreiter
  7. Taiji Suzuki
  8. Yann LeCun

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Dan Hendrycks
  3. Matthias Hein
  4. Dawn Song
  5. Adina Williams
  6. Jörn-Henrik Jacobsen
  7. Alan L. Yuille
  8. Wieland Brendel

Citations in the corpus