A random forest guided tour

By Gérard Biau, Erwan Scornet. 2015

3,741 citations recorded by the source corpus.

The random forest algorithm, proposed by L. Breiman in 2001, has been extremely successful as a general-purpose classification and regression method. The approach, which combines several randomized decision trees and aggregates their predictions by averaging, has shown excellent performance in settings where the number of variables is much larger than the number of observations. Moreover, it is versatile enough to be applied to large-scale problems, is easily adapted to various ad hoc learning tasks, and returns measures of variable importance. The present article reviews the most recent theoretical and methodological developments for random forests. Emphasis is placed on the mathematical forces driving the algorithm, with special attention given to the selection of parameters, the resampling mechanism, and variable importance measures. This review is intended to provide non-experts easy access to the main ideas.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Metalearners for estimating heterogeneous treatment effects using machine learning (2017, background): “For example, Athey and Imbens (2016, 2015) develop Classification and Regression Trees (CART) for CATE estimation, and Wager and Athey (2015) provide asymptotically valid inference methods for causal forests—a variant of Random Forests that is particularly amenable to theoretical analysis because it”
Optimal randomized classification trees (2021, background): “In particular, the two most popular importance measures for forests are outlined in [34]: the Mean Decrease Impurity (MDI) and the Mean Decrease Accuracy (MDA).”
Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability (2021, neutral): “Random forest is a supervised machine learning algorithm that can be used for both regression and classiﬁcation [47].”
Sparsity in Optimal Randomized Classification Trees (2020, background): “Standard CARTs or Random Forests (RFs) [5, 7, 13, 16] cannot manage it due to the greedy construction of the trees.”
Distributional regression forests for probabilistic precipitation forecasting in complex terrain (2018, background): “Applying the Law of Large Numbers it can be shown that random forests do not overfit as the number of trees increases (Breiman 2001; Hastie et al. 2001; Biau and Scornet 2016).”
Estimating Treatment Effects with Causal Forests: An Application (2019, neutral): “We refer to Biau and Scornet (2016) for a recent overview of random forests and note that, throughout, all our forests are “honest” in the sense of Wager and Athey (2018).”
SIRUS: making random forests interpretable (2019, methodology): “In our context of binary classiﬁcation, where the output Y ∈ { 0 , 1 } , maximizing the so-called empirical CART-splitting criterion is equivalent to maximizing the criterion based on Gini impurity (see, e.g., Biau and Scornet, 2016).”
Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution (2018, neutral): “; Biau et al. (2008); Biau (2012); Arlot and Genuer (2014) that, in particular, generate a connection to kernel methods observed in Davies and Ghahramani (2014) and Scornet (2016). See Section 3.1 of Biau and Scornet (2016) for an overview of these developments. More recent results in Wager and Athe”
Double Debiased Machine Learning Nonparametric Inference with Continuous Treatments (2019, neutral): “, Biau and Scornet (2016), Athey, Tibshirani, and Wager (2019), and references therein.”