SOME INFINITY THEORY FOR PREDICTOR ENSEMBLES

By Leo Breiman. 2000

221 citations recorded by the source corpus.

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Analysis of a Random Forests Model (2010, neutral): “1 Introduction”
An Introduction to Boosting and Leveraging (2002, background): “It can be shown [116, 196, 28] that in the infinite sample limit, where the”
Analysis of purely random forests bias (2014, methodology): “Breiman first began to study such simplified RF (Breiman, 2000), and then well-established results were obtained by Biau et al. (2008).”
On the inductive bias of dropout (2014, unknown): “L G ] 1 7 Fe b 20 15”
Tree-Based Batch Mode Reinforcement Learning (2005, methodology): “3 TREE BAGGING We refer here to the standard algorithm published by Breiman (1996). An ensemble of M trees is built.”
Convexity, Classification, and Risk Bounds (2006, background): “In the setting of scaled convex hulls of a base class, some approximation results are given by Breiman (2000), Mannor et al. (2002) and Lugosi and Vayatis (2003).”
Consistency of Random Forests and Other Averaging Classifiers (2008, methodology): “Different random forests differ in how randomness is introduced in the tree building process, ranging from extreme random splitting strategies (Breiman [4], Cutler and Zhao [7]) to more involved data-dependent strategies (Amit and Geman [1], Breiman [5], Dietterich [11]).”
Theory of classification : a survey of some recent advances (2005, background): “It was Breiman [51] who observed that boosting performs gradient descent optimization of an empirical cost function different from the number of misclassified samples, see also Mason, Baxter, Bartlett, and Frean [157], Collins, Schapire, and Singer [61], Friedman, Hastie, and Tibshirani [95].”
On the asymptotics of random forests (2014, methodology): “This feature makes the resulting estimate highly data-dependent, and therefore difficult to analyze (this is particularly the case for Breiman’s (2001) forests). To simplify the analysis, we investigate hereafter infinite random forest estimates whose weights depends only on X,X1, . . . ,Xn which is”
Reconciling modern machine learning and the bias-variance trade-off (2018, background): “Averaging potentially non-smooth interpolating solutions leads to an interpolating solution with a higher degree of smoothness; see [7] for some analysis of infinite ensembles of trees.”
Boosting With the L2 Loss (2003, background): “(13) L2Boosting then yields an estimate F̂m for the unknown function E[Y |X = x] = 2p(x)−1, where p(x) = P[Y = 1|X = x]; the classification rule is given by (5).”
Surprising properties of dropout in deep networks (2016, background): “This abstracts away sampling and optimization issues to focus on the inductive bias, as in [5, 30, 4, 18, 14].”
Fast Binary Feature Selection with Conditional Mutual Information (2004, background): “In this section, recall that we denote byXν(1), . . . , Xν(K) the selected features.”
A random forest guided tour (2015, background): “The connection between forests and kernel estimates is mentioned in Breiman (2000a) and developed in detail in Geurts et al. (2006).”
Understanding Random Forests: From Theory to Practice (2014, methodology): “From this proposition, the consistency of Purely Random Forests [Breiman, 2000] is established.”
Explaining Black-box Android Malware Detection (2018, background): “, nonlinear SVMs may reliably approximate random forests [19].”
On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification (2010, unknown): “This concludes the proof.”