A Hierarchy of Limitations in Machine Learning

By Momin M. Malik. 2020

75 citations recorded by the source corpus.

"All models are wrong, but some are useful", wrote George E. P. Box (1979). Machine learning has focused on the usefulness of probability models for prediction in social systems, but is only now coming to grips with the ways in which these models are wrong---and the consequences of those shortcomings. This paper attempts a comprehensive, structured overview of the specific conceptual, procedural, and statistical limitations of models in machine learning when applied to society. Machine learning modelers themselves can use the described hierarchy to identify possible failure points and think through how to address them, and consumers of machine learning models can know what to question when confronted with the decision about if, where, and how to apply machine learning. The limitations go from commitments inherent in quantification itself, through to showing how unmodeled dependencies can lead to cross-validation being overly optimistic as a way of assessing model performance.

Subfields: Artificial Intelligence, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Scaling & Learning Theory

Citations in the corpus

AI and the Everything in the Whole Wide World Benchmark (2021, background): “…by the machine learning community as a particularly difﬁcult task in benchmark design and development [Mitchell, 2021, Jacobs and Wallach, 2021, Malik, 2020] — however as Bowman and Dahl note, it remains a generally neglected issue as “[t]his criterion is difﬁcult to fully formalize, and we…”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “Without direct measures for these highly complex considerations, AI development teams use proxies, which can create many risks [150].”
On the Machine Learning of Ethical Judgments from Natural Language (2022, background): “In this way, Delphi engages in concept drift (Malik, 2020), by modeling what is operationalizable (text) rather than the concept itself (situations and ethics).”
Leakage and the reproducibility crisis in machine-learning-based science (2023, background): “, without any assumptions about independence in the data) is a hard problem, because we might not know the underlying dependency structure of the task in many cases.(60) [L3.”