Troubling Trends in Machine Learning Scholarship

By Zachary Chase Lipton, Jacob Steinhardt. 2018

367 citations recorded by the source corpus.

Flawed scholarship threatens to mislead the public and stymie future research by compromising ML's intellectual foundations. Indeed, many of these problems have recurred cyclically throughout the history of AI and, more broadly, in scientific research. In 1976, Drew McDermott chastised the AI community for abandoning self-discipline, warning prophetically that "if we can't criticize ourselves, someone else will save us the trouble." The current strength of machine learning owes to a large body of rigorous research to date, both theoretical and empirical. By promoting clear scientific thinking and communication, our community can sustain the trust and investment it currently enjoys.

Subfields: Artificial Intelligence, Information Systems and Management, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems and Management

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring the Algorithmic Efficiency of Neural Networks (2020, methodology): “…search, and appropriate scaling A more thorough study would have carefully ablated all the features of interest from successful models while controlling for model size to be able to attribute the efﬁciency gains to speciﬁc improvements in a quantitative manner [Lipton & Steinhardt, 2018].”
Show Your Work: Improved Reporting of Experimental Results (2019, background): “2186 tained purely through more intensive hyperparameter tuning (Melis et al., 2018; Lipton and Steinhardt, 2018).”
How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks (2018, background): “For more general discussions of misaligned incentives and empirical rigor in machine learning research, we point the interested reader to Lipton and Steinhardt (2018) and Sculley et al.”
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions (2018, background): “Lipton and Steinhardt caution against collapsing statistical parities with legal or ethical concepts [81].”
An overview of deep learning in medical imaging focusing on MRI (2018, background): “Some are general: from technical challenges related to the lack of mathematical and theoretical underpinnings of many central deep learning models and techniques, and the resulting difficulty in deciding exactly what it is that makes one model better than another, to societal challenges related to m”
Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning (2019, background): “Recent critiques of machine learning have already noted a worrying tendency to con-ﬂate speculation and explanation (Lipton & Steinhardt, 2018).”
Pitfalls of Graph Neural Network Evaluation (2018, background): “This makes it difficult to identify whether the improved performance comes from (a) a superior architecture of the new model, or (b) a better-tuned training procedure and / or hyperparameter configuration that unfairly benefits the new model [Lipton and Steinhardt, 2018].”
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift (2019, methodology): “Contributions In the spirit of calls for more rigorous understanding of existing methods (Lipton and Steinhardt, 2018; Sculley et al., 2018; Rahimi and Recht, 2017), this paper provides a benchmark for evaluating uncertainty that focuses not only on calibration in the i.”
A survey and critique of multiagent deep reinforcement learning (2018, background): “28 However, we found very few papers with this characteristic[17,170,208] — we note that this is not encouraged in the ML community; moreover, negative results reduce the chance of paper acceptance [197].”
Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization (2018, background): “Yet, as pointed out by Lipton and Steinhardt (2018), the covariate shift phenomenon itself is not rigorously shown to be the reason behind the performance of Batch Normalization.”
No Training Required: Exploring Random Encoders for Sentence Classification (2019, background): “Progress has been swift, but lately we have started to observe some troubling trends in how research is conducted, in particular with respect to properly identifying the sources of empirical gains (see also Lipton & Steinhardt (2018)).”
A Fair Comparison of Graph Neural Networks for Graph Classification (2019, background): “Over the years, researchers have raised concerns about several ﬂaws in scholarship, such as experimental reproducibility and replicability in machine learning (McDermott, 1976; Lipton & Steinhardt, 2018) and science in general (National Academies of Sciences & Medicine, 2019).”