Random Forests and Adaptive Nearest Neighbors

By Yi Lin, Yongho Jeon. 2006

576 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability, Statistical Learning Theory, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Statistical Learning Theory

In Signal Processing

Citations in the corpus

Generalized random forests (2016, methodology): “This adaptive nearest neighbors perspective also underlies several statistical analyses of random forests, including Arlot and Genuer (2014), Biau and Devroye (2010), and Lin and Jeon (2006).”
Consistency of Random Forests (2014, background): “A critical step was subsequently taken by Lin and Jeon (2006), who established an interesting connection between random forests and a particular class of nearest neighbor predictors, further explored by Biau and Devroye (2010).”
Uniform Convergence of Random Forests via Adaptive Concentration (2015, methodology): “Combining Lemma 12 with our assumptions (28) and (29), we ﬁnd that, or, in other words, all rectangles of size w n must have at least (1 − η/ 2) k n points in them.”
Node harvest: simple and interpretable regression and classication (2009, background): “Lin and Jeon (2006) interpreted Random Forests as an adaptive nearest neighbor scheme, with the distance metric given by the grown tree ensemble.”
Distributional regression forests for probabilistic precipitation forecasting in complex terrain (2018, methodology): “Following Hothorn and Zeileis (2017) we interpret random forests as adaptive local likelihood estimators using the averaged “nearest neighbor weights” (Lin and Jeon 2006) from the T trees in the forest wforesti (z) = 1 T T∑ t=1 Bt∑ b=1 1((zi ∈ Btb) ∧ (z ∈ Btb)) |Btb| , (2.3.1) where |Btb| denotes…”
Explanation and Reliability of Individual Predictions (2013, methodology): “Random forests [8] were rein-terpreted as a weighted mean of the observed response variables in [18] and generalized to Quantile Regression Forests in [19].”
Understanding variable importances in forests of randomized trees (2013, background): “Zhao (2000), Breiman (2004), Biau et al. (2008); Biau (2012), Meinshausen (2006) and Lin and Jeon (2006) investigated simplified to very realistic variants of these algorithms and proved the consistency of those variants.”
Extremely randomized trees (2006, background): “Along a slightly different line of reasoning, Lin and Jeon (2002) show that if the number of samples nti , j in each terminal node of all trees is kept constant and equal to k as N → ∞, then the number of samples which could influence the prediction of an ensemble of trees at some point14 is on the…”
Analysis of purely random forests bias (2014, background): “Few theoretical results exist on RF, mainly the analysis of Bagging of Bühlmann and Yu (2002)—bagging, introduced by Breiman (1996), can be seen a posteriori as a particular case of RF— and the link between RF and nearest neighbors (Lin and Jeon, 2006; Biau and Devroye, 2010).”
Consistency of Random Forests and Other Averaging Classifiers (2008, methodology): “Some attempts to investigate the driving force behind consistency of random forests are by Breiman [4, 6] and Lin and Jeon [14], who establish a connection between random forests and adaptive nearest neighbor methods.”
Comparison of nearest neighbour approaches for small area estimation of tree species-specific forest inventory attributes in central Europe using airborne laser scanner data (2010, background): “However, Lin and Jeon (2006) showed that ensemble methods can also be used to define distances in nearest neighbour methods.”
On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification (2010, methodology): “Proof of the result is due to Lin and Jeon (see [18, Theorem 3, page 581]) and is given here for completeness.”
A new variable importance measure for random forests with missing data (2012, background): “Breiman (2001) proves that random forests don’t overﬁt with a rising number of trees (while the results of Lin and Jeon 2006, indicate that they do overﬁt when trees are grown too large).”
Asymptotic Theory for Random Forests (2014, methodology): “We thus follow the lead of Lin and Jeon [24], and analyze a more general class of predictors— potential nearest neighbors predictors—that operate by doing a nearest-neighbor search over rectangles.”
Consistency of Online Random Forests (2013, methodology): “Two of the models they study are direct simpliﬁcations of the forest growing algorithms used in practice; the others are stylized neighbourhood averaging rules, which can be viewed as simpliﬁcations of random forests through the lens of Lin & Jeon (2002).”