Recursive partitioning for heterogeneous causal effects
By Susan Athey, Guido W. Imbens. 2015
1,679 citations recorded by the source corpus.
In this paper we propose methods for estimating heterogeneity in causal effects in experimental and observational studies and for conducting hypothesis tests about the magnitude of differences in treatment effects across subsets of the population. We provide a data-driven approach to partition the data into subpopulations that differ in the magnitude of their treatment effects. The approach enables the construction of valid confidence intervals for treatment effects, even with many covariates relative to the sample size, and without “sparsity” assumptions. We propose an “honest” approach to estimation, whereby one sample is used to construct the partition and another to estimate treatment effects for each subpopulation. Our approach builds on regression tree methods, modified to optimize for goodness of fit in treatment effects and to account for honest estimation. Our model selection criterion anticipates that bias will be eliminated by honest estimation and also accounts for the effect of making additional splits on the variance of treatment effect estimates within each subpopulation. We address the challenge that the “ground truth” for a causal effect is not observed for any individual unit, so that standard approaches to cross-validation must be modified. Through a simulation study, we show that for our preferred method honest estimation results in nominal coverage for 90% confidence intervals, whereas coverage ranges between 74% and 84% for nonhonest approaches. Honest estimation requires estimating the model with a smaller sample size; the cost in terms of mean squared error of treatment effects for our preferred method ranges between 7–22%.
Subfields: Artificial Intelligence, Statistics and Probability, Causal Inference & Discovery
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Susan Athey
- Stefan Wager
- Mihaela van der Schaar
- Cynthia Rudin
- Nathan Kallus
- Victor Chernozhukov
- Ahmed M. Alaa
- Ioana Bica
In Statistics and Probability
- Stefan Wager
- Susan Athey
- Victor Chernozhukov
- Guido W. Imbens
- Christian Hansen
- Mihaela van der Schaar
- Nathan Kallus
- Ahmed M. Alaa
In Causal Inference & Discovery
- Susan Athey
- Mihaela van der Schaar
- Stefan Wager
- Ahmed M. Alaa
- Guido W. Imbens
- Victor Chernozhukov
- Ioana Bica
- Edward H. Kennedy
Citations in the corpus
- Machine Learning Methods That Economists Should Know About (2019, methodology): “The panel data literature traditionally focused on fixed effect methods, and has generalized those to models with multiple latent factors (Bai and Ng [2017, 2002], Bai [2003]) that are essentially the same as the low rank factorizations in the ML literature.”
- Deep-Treat: Learning Optimal Personalized Treatments From Observational Data Using Neural Networks (2018, background): “There are many works in estimating Individualized Treatment Effects (ITE) (Athey and Imbens 2015; Hill 2011).”
- Local Linear Forests (2018, methodology): “To compare the performance of both methods, we use the transformed outcome metric of Athey and Imbens (2016).”
- Generalized random forests (2016, methodology): “The dominant bias term in err (C1, C2) is due to the sampling variance of regression trees, and is the same term that appears in the analysis of Athey and Imbens (2016).”
- The Econometrics of Randomized Experiments (2016, methodology): “1, we discuss machine learning methods for partitioning the covariate space according to treatment effect heterogeneity; if we construct indicators for the element of the partition derived according to an “honest causal tree” (Athey and Imbens, 2016) and incorporate them into (5.4), then the resulti”
- Estimation and Inference of Heterogeneous Treatment Effects using Random Forests (2015, methodology): “In growing trees to build our forest, we follow most closely the approach of Athey and Imbens [2016], who propose honest, causal trees, and obtain valid confidence intervals for average treatment effects for each of the subpopulations (leaves) identified by the algorithm. (Instead of personalizing p”
- Deep Counterfactual Networks with Propensity-Dropout (2017, background): “…of inferring individualized treatment effects from observational datasets is a fundamental problem in many domains such as precision medicine (Shalit et al., 2017), econometrics (Abadie & Imbens, 2016), social sciences (Athey & Imbens, 2016), and computational advertising (Bottou et al., 2013).”
- Learning When-to-Treat Policies (2019, background): “3See also Athey and Imbens (2018) for a closely related discussion of potential outcomes in the context of staggered adoption.”
- Causal Falling Rule Lists (2015, methodology): “Methods have used tree structures (Su, Kang, and Levine 2012; Su and Nickerson 2009; Athey and Imbens 2015; Beygelzimer and Langford 2009) to form such treatment effect subgroups.”
- Efficient Policy Learning (2017, background): “There has also been some recent interest in developing tree-based decision rules (Athey and Imbens, 2016; Kallus, 2017; Su et al., 2009).”
- Automated versus Do-It-Yourself Methods for Causal Inference: Lessons Learned from a Data Analysis Competition (2017, background): “Variants of this have been proposed that stratify instead within leaves of a regression tree (Athey and Imbens, 2016; Wager and Athey, 2015).”
- Quasi-oracle estimation of heterogeneous treatment effects (2017, methodology): “Finally, it is straightforward to tune the regularizer Λ n ( τ ) by simply cross validating the squared-error loss in (4), which avoids the use of more sophisticated model-assisted cross-validation procedures as developed in Athey and Imbens (2016) or Powers et al. (2018).”
- Towards optimal doubly robust estimation of heterogeneous causal effects (2020, background): “The last 5–10 years has seen even more emphasis on nonpara-metrics and incorporating machine learning [Athey and Imbens, 2016, Foster and Syrgkanis, 2019, Foster et al., 2011, Hahn et al., 2020, Imai and Ratkovic, 2013, Künzel et al., 2019, Nie and Wager, 2021, Semenova and Chernozhukov, 2017,…”
- Estimating Average Treatment Effects: Supplementary Analyses and Remaining Challenges (2017, methodology): “See Athey and Imbens [2015a] for details. We also report summary statistics of b̂(Xi), the average, the median and the 0.025, 0.25, 0.75 and 0.975 quantiles, based on random forest methods. We also present a histogram of b̂(x)fs. For the Connors et al. [1996] data the methods do vary substantially, ”
- GENERATIVE ADVERSARIAL NETS (2018, methodology): “More recent works focus on individualized treatment effects (Chipman et al. (2010); Wager & Athey (2017); Athey & Imbens (2016); Lu et al. (2017); Alaa & van der Schaar (2017); Porter et al. (2011); Johansson et al. (2016); Alaa et al. (2017); Louizos et al. (2017); Shalit et al. (2017)).”