An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

By Thomas G. Dietterich. 2000

3,138 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

Analysis of a Random Forests Model (2010, methodology): “Breiman’s ideas were decisively inﬂuenced by the early work of Amit and Ge-man [3] on geometric feature selection, the random subspace method of Ho [27] and the random split selection approach of Dietterich [21].”
Random Forests (2001, methodology): “Following Dietterich (1998) the following experiment was done which changed about one in twenty class labels (injecting 5% noise).”
Generalized random forests (2016, methodology): “In standard classification or regression forests as proposed by Breiman (2001), the prediction for a particular test point x is determined by averaging predictions across an ensemble of different trees (Amit and Geman, 1997; Breiman, 1996; Dietterich, 2000; Ho, 1998).”
Ensemble based systems in decision making (2006, background): “For example, Dietterich [66], [67], Breiman [68], Bauer et al.”
Ensemble machine learning on gene expression data for cancer classification. (2003, methodology): “Such direct optimisation may lead to the risk of overfitting because the hypothesis space for the ensemble is much larger than the original algorithm (Dietterich 2000a).”
Spatial disaggregation of complex soil map units: A decision-tree based approach in Bavarian forest soils (2012, methodology): “Random forest is an ensemble method in which many different classification trees are combined to produce a more stable and accurate classification compared to a single decision tree (Bauer and Kohavi, 1999; Breiman, 1996; Dietterich, 2000).”
Multi-Class Deep Boosting (2014, background): “And indeed, overfitting has been reported in practice for AdaBoost in the past [Grove and Schuurmans, 1998, Schapire, 1999, Dietterich, 2000, Rätsch et al., 2001b].”
Boosting With the L2 Loss (2003, methodology): “This adds insights to the known empirical fact that boosting doesn’t work well in noisy problems, see Dietterich (2000).”
A Review of Ensemble Methods in Bioinformatics (2010, background): “These ensemble techniques have the advantage to alleviate the small sample size problem by averaging and incorporating over multiple classification models to reduce the potential for overfitting the training data [9].”
Boosting Algorithms as Gradient Descent in Function Space (2007, methodology): “Grove and Schuurmans [15] demonstrated that running AdaBoost for hundreds of thousands of rounds can lead to signi cant over tting, while a number of authors [6, 19, 2, 16] showed that, by adding label noise, over tting can be induced in AdaBoost even with relatively few classi ers in the combinatio”
Evaluation of random forest and adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery (2008, methodology): “Both methods have been thoroughly studied (Bauer & Kohavi, 1998; Dietterich, 2000) and have been assessed with remote sensing data (Chan et al., 2001; DeFries & Chan, 2000; Pal & Mather, 2003).”
Forecasting cancellation rates for services booking revenue management using data mining (2010, methodology): “Random Forest (RF) (Breiman 2001) combines several successful techniques in building tree ensembles such as bagging (Breiman 1996) and random split selection (Dietterich 2000) to construct a collection of decision trees.”
Round Robin Classification (2002, methodology): “…subsets (Bay, 1999), to exploit the randomness of the base algorithms (Kolen and Pollack, 1991), possibly by artiﬁcially randomizing their behavior (Dietterich, 2000b), or to use multiple representations of the domain objects, for example by using information originating from diﬀerent hyperlinks…”