Adaptive Concentration of Regression Trees, with Application to Random Forests

By Stefan Wager, Guenther Walther. 2015

149 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

Local Linear Forests (2018, background): “Biau, Devroye, and Lugosi (2008) and Wager and Walther (2015) relied on large leaves, while the results of Scornet, Biau, and Vert (2015) on fully grown trees rely on an unchecked high-level assumption.”
Generalized random forests (2016, background): “…consistency (Arlot and Genuer, 2014; Biau, Devroye and Lugosi, 2008; Biau, 2012; Denil, Matheson and De Freitas, 2014; Lin and Jeon, 2006; Scornet, Biau and Vert, 2015; Wager and Walther, 2015), secondorder asymptotics (Mentch and Hooker, 2016), and confidence intervals (Wager and Athey, 2018).”
The Econometrics of Randomized Experiments (2016, background): “Other related approaches include Wager and Walther (2015), who discuss corrections to confidence intervals (widening the confidence intervals by a factor) as an alternative to sample splitting; however, since confidence intervals need to be inflated fairly substantially, it is not clear whether…”
Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes (2019, methodology): “Eschewing such conditions allows us to use such estimators as the highly adaptive LASSO (Benkeser and van der Laan, 2016), càdlàg function estimators in very high dimensions (Bibaut and van der Laan, 2019b), and random forests (Wager and Walther, 2016) as long as their convergence rates are ensured ”
Nonparametric estimation of causal heterogeneity under high-dimensional confounding (2019, result): “Similar rates can be shown for L2 boosting (Luo and Spindler, 2016) and nonlinear models like Random Forests (Wager and Walther, 2016) or forms of Deep Neural Nets (Farrell, Liang, and Misra, 2018).7 A natural question is then if a bandwidth exists that satisfies the rate conditions in Assumption 8.”
Optimality of Matched-Pair Designs in Randomized Controlled Trials (2022, background): “…rates, it is satisfied by the LASSO estimator (Bühlmann and van de Geer, 2011; Belloni et al., 2014), regression trees and random forests (Györfi et al., 2002; Wager and Walther, 2015), neural nets (White, 1990; Farrell et al., 2018), and support vector machines (Steinwart and Christmann, 2008).”
Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution (2018, unknown): “In comparison, Wager and Walther (2015) have shown that, in ﬁxed dimension, any tree can be well approximated by a collection of O (exp(log n ) 2 ) hyper rectangles.”
: "Statistical Modeling: The Two Cultures" (Statistical Science, 2001, 16(3), 199-231) (unknown): “It has made strides in understanding theoretical aspects of random forests, (Scornet et al., 2015; Wager and Walther, 2015; Athey et al., 2019) and boosting (B¨uhlmann and Yu, 2003; Zhang and Yu, 2005), in understanding prediction accuracy from scratch by establishing non-asymptotic prediction…”
Double/debiased machine learning for treatment and structural parameters (2017, unknown): “…in a variety of sparse models; Kozbur (2016) for forward selection in sparse models; Luo and Spindler (2016) for L2-boosting in sparse linear models; Wager and Walther (2016) for concentration results for a class of regression trees and random forests; and Chen and White (1999) for a class of…”
Optimization of Tree Ensembles (2017, background): “Recently, Wager and Walther (2015) showed that certain forms of regression trees and random forests converge uniformly over the feature space to the true regression function, while Wager and Athey (2015) considered how to use random forests for causal inference.”