Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
By Stephan Rabanser, Stephan Günnemann, Zachary Chase Lipton. 2018
473 citations recorded by the source corpus.
We might hope that when faced with unexpected inputs, well-designed software systems would fire off warnings. Machine learning (ML) systems, however, which depend strongly on properties of their inputs (e.g. the i.i.d. assumption), tend to fail silently. This paper explores the problem of building ML systems that fail loudly, investigating methods for detecting dataset shift, identifying exemplars that most typify the shift, and quantifying shift malignancy. We focus on several datasets and various perturbations to both covariates and label distributions with varying magnitudes and fractions of data affected. Interestingly, we show that across the dataset shifts that we explore, a two-sample-testing-based approach, using pre-trained classifiers for dimensionality reduction, performs best. Moreover, we demonstrate that domain-discriminating approaches tend to be helpful for characterizing shifts qualitatively and determining if they are harmful.
Subfields: Artificial Intelligence, Evaluation & Benchmarks, Anomaly & Outlier Detection, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Bernhard Schölkopf
- Percy Liang
- Kyunghyun Cho
- Quoc V. Le
- Jianfeng Gao
- Sergey Levine
In Evaluation & Benchmarks
- Bernhard Schölkopf
- Bowen Zhou
- Andrei Paleyes
- Leo Anthony Celi
- Bo Li
- Shira Mitchell
- Emma Pierson
- Neil D. Lawrence
In Anomaly & Outlier Detection
In Signal Processing
- Stefan Bauer
- Bernhard Schölkopf
- Anirudh Goyal
- Yoshua Bengio
- Nal Kalchbrenner
- Nan Rosemary Ke
- Francesco Locatello
- Bowen Zhou
Citations in the corpus
- Toward Causal Representation Learning (2021, background): “It also led to a range of follow-up work [32], [78], [97], [114], [115], [152], [153], [156], [167], [195], [204], [243], [263], [267], [277], [278], [281], complementing the studies of Bareinboim and Pearl [14], [185], and it inspired a thread of work in the statistics community exploiting invarian”
- Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, methodology): “However, ML models can perform poorly with such a shift in input variables [111].”
- Causality for Machine Learning (2019, neutral): “er fruitful (Zhang et al., 2013; Weichwald et al., 2014; Zhang et al., 2015; Gong et al., 2016; Huang et al., 2017; Blöbaum et al., 2016; Guo et al., 2018; Wang et al., 2019; Subbaswamy et al., 2018; Rabanser et al., 2018; Li et al., 2018a; Lipton et al., 2018; Li et al., 2018b; Magliacane et al., 2”
- Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics (2020, background): “…catastrophic forgetting, it doesn’t directly represent the scenario where catastrophic forgetting arises through input distribution shift — the input data to the neural network undergoes a distribution shift, causing the neural network to perform poorly on earlier data distributions [12, 9, 10].”
- Trustworthy AI: From Principles to Practices (2021, background): “Data drift monitoring [268] provides important means to maintain the generalization of an AI system under concept change [394] caused by dynamic environment such as market change [289].”
- Distilling Model Failures as Directions in Latent Space (2022, unknown): “…implications for machine learning models’ behavior (Feldman, 2019; Carlini et al., 2019; Koh & Liang, 2017; Ghorbani & Zou, 2019; Ilyas et al., 2022), especially as the training environments often deviate from deployment conditions (Rabanser et al., 2019; Koh et al., 2020; Hendrycks et al., 2020).”
- Ethical Machine Learning in Healthcare (2020, methodology): “Several algorithms have recently been proposed to account for distribution shift in data (135, 136).”
- Challenges in Deploying Machine Learning: A Survey of Case Studies (2020, background): “[110], and are now implemented in software libraries (e.”
- Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare (2022, methodology): “For complex data types like physiological waveforms, medical images, and clinical notes, representation learning methods can transform the data into a lower-dimensional vector that is suitable for inputting into traditional control charts 40,41 .”
- Algorithmic Fairness: Choices, Assumptions, and Definitions (2021, background): “However, models can perform poorly with such a shift in covariates (Rabanser et al. 2018).”
- Towards Robust and Reliable Algorithmic Recourse (2021, background): “Choice of ∆ Predictive models deployed in the real world are often updated regularly to handle data distribution shifts [22].”