Asymptotic Theory for Random Forests

By Stefan Wager. 2014

67 citations recorded by the source corpus.

Random forests have proven to be reliable predictive algorithms in many application areas. Not much is known, however, about the statistical properties of random forests. Several authors have established conditions under which their predictions are consistent, but these results do not provide practical estimates of random forest errors. In this paper, we analyze a random forest model based on subsampling, and show that random forest predictions are asymptotically normal provided that the subsample size s scales as s(n)/n = o(log(n)^{-d}), where n is the number of training examples and d is the number of features. Moreover, we show that the asymptotic variance can consistently be estimated using an infinitesimal jackknife for bagged ensembles recently proposed by Efron (2014). In other words, our results let us both characterize and estimate the error-distribution of random forest predictions, thus taking a step towards making random forests tools for statistical inference instead of just black-box predictive algorithms.

Subfields: Environmental Engineering, Management, Monitoring, Policy and Law, Nature and Landscape Conservation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Environmental Engineering

In Management, Monitoring, Policy and Law

In Nature and Landscape Conservation

Citations in the corpus

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests (2015, methodology): “In our technical report [Wager, 2014], we describe a simple example where μ(x) is constant yet adaptive CART-based random forests are inconsistent.”
Consistency of Random Forests (2014, background): “Thus, as highlighted by Wager (2014), random forests can be inconsistent at some fixed point x ∈ [0, 1]d, particularly near the edges, while being L2 consistent.”
Uniform Convergence of Random Forests via Adaptive Concentration (2015, background): “Methods for doing statistical inference on random forest predictions are also available [31, 38, 43, 44].”
Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests (2014, result): “We would also like to acknowledge similar work currently in progress by Wager (2014).”
On the asymptotics of random forests (2014, methodology): “Recent attempts to bridge the gap between theoretical forest models and Breiman’s (2001) forests have been made by Wager (2014) and Scornet et al. (2014) who establish consistency of the original algorithm under suitable assumptions.”