Prediction Policy Problems
By Jon M. Kleinberg, Jens Ludwig, Sendhil Mullainathan, Z. Obermeyer. 2015
586 citations recorded by the source corpus.
Most empirical policy work focuses on causal inference. We argue an important class of policy problems does not require causal inference but instead requires predictive inference. Solving these "prediction policy problems" requires more than simple regression techniques, since these are tuned to generating unbiased estimates of coefficients rather than minimizing prediction error. We argue that new developments in the field of "machine learning" are particularly useful for addressing these prediction problems. We use an example from health policy to illustrate the large potential social welfare gains from improved prediction.
Subfields: Economics and Econometrics, Statistics and Probability, Economics & Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Economics and Econometrics
- Susan Athey
- Stefan Wager
- Jens Ludwig
- Edgar Dobriban
- Sendhil Mullainathan
- Jon M. Kleinberg
- Momin M. Malik
- Ravi Shroff
In Statistics and Probability
- Susan Athey
- Stefan Wager
- Edward H. Kennedy
- Cynthia Rudin
- Edgar Dobriban
- Sendhil Mullainathan
- Guido W. Imbens
- Jens Ludwig
In Economics & Operations Research
- Stefan Wager
- Susan Athey
- Edgar Dobriban
- Momin M. Malik
- Jon M. Kleinberg
- Sendhil Mullainathan
- Alexandra Chouldechova
- Daniel G. Goldstein
Citations in the corpus
- Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment (2017, background): “might unduly push institutions to reframe their goals in a way that is amenable to machine learning analysis [47].”
- Predictive Multiplicity in Classification (2019, background): “Many models are now built merely to make accurate predictions [24, 45].”
- Explainable machine learning in deployment (2019, background): “The relationship between causality and explainability in MLrelatestoabroaderdichotomy.IndistinguishingMLtechniquesfrom those in econometrics or statistics, a key component is prediction vs. inference [10, 11, 33]. ML focuses on prediction, such that understandingtheunderlyingprocessisgenerallyconsid”
- The State of Applied Econometrics - Causality and Policy Evaluation (2016, neutral): “Kleinberg et al. [2015] argue that many important policy problems are fundamentally prediction problems; see also the review article in this volume.”
- Estimation and Inference of Heterogeneous Treatment Effects using Random Forests (2015, background): “For example, Kleinberg et al. [2015] seek to improve the allocation of medicare funding for hip or knee replacement surgery by detecting patients who had been prescribed such a surgery, but were in fact likely to die of other causes before the surgery would have been useful to them.”
- Simple Rules for Complex Decisions (2017, background): “A large body of work shows that intuitive judgments are generally inferior to those based on statistical models [4, 5, 19, 20, 33].”
- The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence (2022, background): “Automated decision-making systems (ADS) are increasingly used to make rapid and broad decisions about people [6, 15, 16], creating the possibility of negative unintended consequences that often affect the very communities that are already marginalized, underserved, or underrepresented [7, 3].”
- Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making (2021, background): “However, it remains difficult to enhance human decision making in challenging prediction tasks [28].”
- On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection (2018, background): “Moreover, since many policy decisions can be formulated as prediction problems [37], understanding human predictions with assistance from machine learning models”
- A Hierarchy of Limitations in Machine Learning (2020, background): “But for pre-testing the design of a system for detection/classification by a given schema (especially if developed by or with relevant stakeholders), representing an arguably enormous space of use cases neglected prior to the development of machine learning (Breiman, 2001; Mullainathan and Spiess, 2”
- High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification (2015, methodology): “, 2014) to Medicare resource allocation (Kleinberg et al., 2015).”
- Counterfactual risk assessments, evaluation, and fairness (2019, background): “Models trained and evaluated in this way answer the question: What is the likelihood of an adverse outcome under the observed historical decision process? Yet the question relevant to the decision maker is: What is the likelihood of an adverse outcome under the proposed decision? When decisions do n”