Random Forests
By Trevor J. Hastie, Leo Breiman, Robert Tibshirani, Jerome H. Friedman. 2001
119,944 citations recorded by the source corpus.
Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, ***, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Quoc V. Le
- Sergey Levine
- Jianfeng Gao
- Ruslan Salakhutdinov
- Trevor Darrell
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
- Yi Yang
In Model Architectures
- Yoshua Bengio
- Quoc V. Le
- Ruslan Salakhutdinov
- Kaiming He
- Alan L. Yuille
- Ross B. Girshick
- Aaron C. Courville
- Kyunghyun Cho
Citations in the corpus
- Improving neural networks by preventing co-adaptation of feature detectors (2012, methodology): “Bagging is most often used with models such as decision trees because these are very quick to fit to data and very quick at test time[3].”
- Analysis of a Random Forests Model (2010, methodology): “As highlighted by various empirical studies (see [11, 36, 20, 24, 25] for instance), random forests have emerged as serious competitors to state-of-the-art methods such as boosting (Freund [22]) and support vector machines (Shawe-Taylor and Cristianini [35]).”
- Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets (2015, background): “History shows that there are two major sources of breakthroughs: scaling up the existing approaches to larger datasets, and development of novel, more accurate approaches (LeCun et al., 1998; Elman, 1990; Breiman, 2001).”
- Challenges in representation learning: A report on three machine learning contests (2013, methodology): “David Thaler won the contest with an accuracy of 70.22% using blending of three models that used sparse filtering[4] for feature learning, random forests for feature selection [5], and support vector machines[6] for classification.”
- Reconciling modern machine-learning practice and the classical bias–variance trade-off (2018, methodology): “We train standard random forests introduced by Breiman [9] for regression problems.”
- Fast Edge Detection Using Structured Forests (2014, methodology): “Throughout our presentation we adopt the notation and terminology of the extensive recent survey by Criminisi et al. [11], somewhat simplified for ease of presentation.”
- Analyzing the Performance of Multilayer Neural Networks for Object Recognition (2014, methodology): “regression was not used for detection results.) SUN-CLS PASCAL-DET PASCAL-DET+DATA scratch pre-train ne-tune scratch pre-train ne-tune scratch pre-train ne-tune initialization) on the detection data? (2) How does the amount of ne-tuning35:7 0:2 48:4 0:1 52:2 0:1 40.7 45.5 54.1 52.3 45.5 59.2 data ch”
- Learning Rich Features from RGB-D Images for Object Detection and Segmentation (2014, methodology): “We opt for the first option and work with random forests [5], which naturally deal with multi-modal data and have been shown to work well with the set of features we have designed [27,31].”
- Crowdsourcing Multiple Choice Science Questions (2017, methodology): “As classifier we chose a random forest (Breiman, 2001), because of its robust performance in small and mid-sized data settings and its power to incorporate nonlinear feature interactions, in contrast, e.g., to logistic regression.”
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (2016, methodology): “Broadly, there are two classes of ensembles: randomisation -based approaches such as random forests (Breiman, 2001), where the ensemble members can be trained in parallel without any interaction, and boosting -based approaches where the ensemble members are fit sequentially.”
- Practical Evasion of a Learning-Based Classifier: A Case Study (2014, methodology): “PDFRATE employs Random Forest [31], an ensemble learning method comprising a number tRF of independently trained decision trees.”
- Bayesian Nonparametric Federated Learning of Neural Networks (2019, methodology): “Ensemble methods (Dietterich, 2000; Breiman, 2001) are a classic approach for combining predictions of multiple learners.”
- FairSight: Visual Analytics for Fairness in Decision Making (2019, methodology): “We adopt the feature perturbation method [5], a widely-used feature auditing technique to evaluate the feature impact on a model.”
- Interpreting Classifiers through Attribute Interactions in Datasets (2017, background): “ of the proposedmethod. 1. Introduction A lot of attention has been on creating high-performing classifiers such as, e.g., support vector machines (SVMs) (Cortes & Vapnik, 1995) and random forest (Breiman, 2001), both of which are among the best-performing classifiers (Ferna´ndez-Delgadoet al., 20”