Generalized random forests

By Susan Athey, Julie Tibshirani, Stefan Wager. 2016

1,749 citations recorded by the source corpus.

We propose generalized random forests, a method for non-parametric statistical estimation based on random forests (Breiman, 2001) that can be used to fit any quantity of interest identified as the solution to a set of local moment equations. Following the literature on local maximum likelihood estimation, our method considers a weighted set of nearby training examples; however, instead of using classical kernel weighting functions that are prone to a strong curse of dimensionality, we use an adaptive weighting function derived from a forest designed to express heterogeneity in the specified quantity of interest. We propose a flexible, computationally efficient algorithm for growing generalized random forests, develop a large sample theory for our method showing that our estimates are consistent and asymptotically Gaussian, and provide an estimator for their asymptotic variance that enables valid confidence intervals. We use our approach to develop new methods for three statistical tasks: non-parametric quantile regression, conditional average partial effect estimation, and heterogeneous treatment effect estimation via instrumental variables. A software implementation, grf for R and C++, is available from CRAN.

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, background): “[21] proposed a bootstrap-of-little-bags approach to estimate the variance of functions learned from bagged ensembles, and Athey et al. proved the consistency of this variance estimate [4].”
Local Linear Forests (2018, methodology): “…linear forests for nonparametric regression; however, a similar local linear correction can also be applied to quantile regression forests (Meinshausen 2006), causal forests (Wager and Athey 2018) or, more broadly, to any instance of generalized random forests (Athey, Tibshirani, and Wager 2019).”
Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, background): “More recently, Athey et al. (2019) consider this issue in the context of generalized random forests.”
Estimation Considerations in Contextual Bandits (2017, methodology): “In this paper, we consider non-parametric estimation of μa(x) by training a generalized random forest [9] on the history of observations assigned to a.”
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests (2015, positive): “8The R package grf [Athey et al., 2016] provides a newer, high-performance implementation of causal forests, available on CRAN.”
Learning When-to-Treat Policies (2019, methodology): “13For this purpose, we estimate propensities and conditional response surfaces using regression forests as implemented in grf (Athey et al., 2019).”
Efficient Policy Learning (2017, methodology): “…average treatment effect estimation with improved finite sample performance: examples include double machine learning (Belloni et al., 2017; Chernozhukov et al., 2016), targeted learning (van der Laan and Gruber, 2010; van der Laan and Rose, 2011), and residual balancing (Athey et al., 2016a).”
Quasi-oracle estimation of heterogeneous treatment effects (2017, background): “For example, Zhao, Small, and Ertefaie (2017) use Robinson’s transfomation for post-selection inference on eﬀect modiﬁcation; Athey, Tibshirani, and Wager (2018) rely on it to grow a causal forest that is robust to confounding; Robins (2004) builds on it in developing G -estimation for sequential…”
Double Debiased Machine Learning Nonparametric Inference with Continuous Treatments (2019, neutral): “Our DML estimator serves as the preliminary element for policy learning and optimization with a continuous decision, following Manski (2004), Hirano and Porter (2009), Kitagawa and Tetenov (2018), Kallus and Zhou (2018), Demirer et al. (2019), Athey and Wager (2019), among others.”
Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects (with Discussion) (2020, neutral): “In Section 6.1 we compare BCF, generalized random forests (Athey et al., 2019), and a linear model with all three-way interactions as plausible methods for estimating heterogeneous treatment effects with measures of uncertainty.”
Offline Multi-Action Policy Learning: Generalization and Optimization (2018, methodology): “This quantity can be conveniently estimated using the generalized random forest approach developed in Athey et al. (2016b), which also provides a free software package.”
Representation Learning for Treatment Effect Estimation from Observational Data (2018, neutral): “Besides, some tree and forest based methods [7, 33, 4, 3] view the tree and forests as an adaptive neighborhood metric, and estimate the treatment effect at the leaf node.”
Interpretable Almost-Exact Matching for Causal Inference (2019, methodology): “For Causal Forest, we used the ‘grf’ R-package (Athey et al., 2019).”
Conformal inference of counterfactuals and individual treatment effects (2020, methodology): “We further estimate the conditional quantiles in three different ways: (1) via quantile random forest (Athey et al., 2019) by using the grf package (Tibshirani et al.”
Nonparametric estimation of causal heterogeneity under high-dimensional confounding (2019, background): “Athey, Tibshirani, and Wager (2019) and Lechner (2018) modify the Random Forest algorithm to better adjust for confounding and improve precision.”
A Brief Review of Random Forests for Water Scientists and Practitioners and Their Recent History in Water Resources (2019, background): “, references [89,95]); (4) distributional regression forests for estimation of the location, scale, and shape distribution parameters (i.”