On the asymptotics of random forests

By Erwan Scornet. 2014

107 citations recorded by the source corpus.

The last decade has witnessed a growing interest in random forest models which are recognized to exhibit good practical performance, especially in high-dimensional settings. On the theoretical side, however, their predictive power remains largely unexplained, thereby creating a gap between theory and practice. In this paper, we present some asymptotic results on random forests in a regression framework. Firstly, we provide theoretical guarantees to link finite forests used in practice (with a finite number M of trees) to their asymptotic counterparts (with M = ∞ ). Using empirical process theory, we prove a uniform central limit theorem for a large class of random forest estimates, which holds in particular for Breiman's (2001) original forests. Secondly, we show that infinite forest consistency implies finite forest consistency and thus, we state the consistency of several infinite forests. In particular, we prove that q quantile forests-close in spirit to Breiman's (2001) forests but easier to study-are able to combine inconsistent trees to obtain a final consistent prediction, thus highlighting the benefits of random forests compared to single trees.

Subfields: Artificial Intelligence, Mathematical Physics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Mathematical Physics

In Statistics and Probability

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, background): “The main 1 3 exponent of bagging is Random Forests (Biau and Scornet 2016; Breiman 2001; Fawagreh et al.”
Consistency of Random Forests (2014, methodology): “CONSISTENCY OF RANDOM FORESTS(1) By Erwan Scornet∗, Gérard Biau∗ and Jean-Philippe Vert† Sorbonne Universités∗ and MINES ParisTech, PSL-Research University† Random forests are a learning algorithm proposed by Breiman [Mach. Learn. 45 (2001) 5–32] that combines several randomized decision trees and a”
Uniform Convergence of Random Forests via Adaptive Concentration (2015, background): “Ever since their introduction by Breiman [8], many authors have studied the theoretical properties of random forests in order to explain their performance [1, 4, 5, 9, 16, 28, 30, 35, 36, 37, 43].”
Adaptive Concentration of Regression Trees, with Application to Random Forests (2015, background): “Several authors have studied the theoretical properties of trees and forests in order to explain their performance [1, 5, 6, 9, 14, 19, 28, 31, 33, 37, 38, 39, 45].”
Prediction of the Strength of Rubberized Concrete by an Evolved Random Forest Model (2019, methodology): “Nowadays, the random forest (RF) approach has been employed in predicting the mechanical parameters of concrete due to its excellent performance on nonlinear regression and classification [28, 29].”
A random forest guided tour (2015, methodology): “This rate is minimax over the class of Lipschitz functions (Stone 1980, 1982). It is often acknowledged that random forests reduce the estimation error of a single tree, while maintaining the same approximation error. In this respect, Biau (2012) argues that the estimation error of centered forests ”