Consistency of Random Forests and Other Averaging Classifiers

By Gérard Biau, Luc Devroye, Gábor Lugosi. 2008

624 citations recorded by the source corpus.

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Analysis of a Random Forests Model (2010, background): “…(see also [5] for further results); Meinshausen [33], who studies the consistency of random forests in the context of conditional quantile prediction; and Devroye et al. [6], who oﬀer consistency theorems for various simpliﬁed versions of random forests and other randomized ensemble predictors.”
Mathematical optimization in classification and regression trees (2021, background): “The reader is referred to Biau et al. (2008), Denil et al. (2013), Scornet (2016), and Scornet et al. (2015) for some results in this line.”
Local Linear Forests (2018, neutral): “Biau, Devroye, and Lugosi (2008) and Wager and Walther (2015) relied on large leaves, while the results of Scornet, Biau, and Vert (2015) on fully grown trees rely on an unchecked high-level assumption.”
Generalized random forests (2016, methodology): “Several theoretical results are available on the asymptotic behavior of variants of such forestbased estimates μ̂(x), including consistency (Arlot and Genuer, 2014; Biau et al., 2008; Biau, 2012; Denil et al., 2014; Lin and Jeon, 2006; Scornet et al., 2015; Wager and Walther, 2015) and asymptotic no”
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests (2015, background): “The convergence and consistency properties of trees and random forests have been studied by, among others, Biau [2012], Biau et al. [2008], Breiman [2004], Breiman et al. [1984], Meinshausen [2006], Scornet et al. [2015], and Wager and Walther [2015].”
Uniform Convergence of Random Forests via Adaptive Concentration (2015, background): “Ever since their introduction by Breiman [8], many authors have studied the theoretical properties of random forests in order to explain their performance [1, 4, 5, 9, 16, 28, 30, 35, 36, 37, 43].”
Adaptive Concentration of Regression Trees, with Application to Random Forests (2015, background): “Several authors have studied the theoretical properties of trees and forests in order to explain their performance [1, 5, 6, 9, 14, 19, 28, 31, 33, 37, 38, 39, 45].”
On Oblique Random Forests (2011, background): “They often do well with little parameter tuning [16], and are able to identify relevant feature subsets even in the presence of a large amount of irrelevant predictors [6, 21, 25, 2].”
Analysis of purely random forests bias (2014, methodology): “Breiman first began to study such simplified RF (Breiman, 2000), and then well-established results were obtained by Biau et al. (2008).”
Random Forests for Big Data (2015, methodology): “It would be of interest to use in MR-RF this RF variant, or even more randomized ones (like [29] PERT, Perfect Random Tree Ensembles, or [30, 31] PRF, Purely Random Forests).”
Alternating Decision Forests (2013, background): “Existing explanations in the literature point to comparisons with nearest neighbor algorithms, rules of large numbers [5], classifier consistency [3], and large-margin methods [9], among others.”
A review on machine learning principles for multi-view biological data integration (2016, background): “Recent studies also prove that not all random forest models are universally consistent [55, 56].”
Classification of Imbalanced Marketing Data with Balanced Random Sets (2009, background): “Keywords: ensemble classifier, gradient-based optimisation, boosting, random forests, decision trees”
Practical Ensemble Classification Error Bounds for Different Operating Points (2013, background): “An automatic classifier that identifies chatter instances containing terrorist conversations with modest false alarm probability would overwhelm investigators who need to focus their attention on definite instances; a classifier with ultralow false alarm probability is thus appropriate.”
Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife (2013, background): “Several variants of random forests have been analyzed theoretically by, e.g., Biau et al. (2008), Biau (2012), Lin and Jeon (2006), and Meinshausen (2006).”
A random forest guided tour (2015, background): “In recent years, various theoretical studies have been performed (e.g., Meinshausen 2006; Biau et al. 2008; Ishwaran and Kogalur 2010; Biau 2012; Genuer 2012; Zhu et al. 2015), analyzing more elaborate models and moving ever closer to the practical situation.”
Mining data with random forests: A survey and results of new tests (2011, methodology): “[11] is a combination of the random subspace method proposed by Ho [12] and bagging.”
Asymptotic Theory for Random Forests (2014, background): “As far as we know, prior papers on the consistency of random forests [e.g., 2, 3, 25] all require a similar condition.”
Ensemble Trees and CLTs: Statistical Inference for Supervised Learning (2014, positive): “Biau et al. [2008] restrict their attention to classiﬁcation, but prove consistency of certain bagging and random forest estimators, provided the individual trees are consistent.”