Bias, Variance , And Arcing Classifiers

By Leo Breiman. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning privately from multiparty data (2016, methodology): “For the purpose of analysis, assume that the local classifiers h1(x), ..., hM (x) are from the same hypothesis class.2 Since the local training data are i.i.d. samples from P (x, y), the local classifiers {h1(x), ..., hM (x)} can be considered random hypotheses, as in (Breiman, 1996a).”
Boosting the margin: A new explanation for the effectiveness of voting methods (1997, methodology): “These error curves reveal a remarkable phenomenon, first observed by Drucker and Cortes [8], and later by Quinlan [15] and Breiman [5].”
Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [Review Article] (2016, background): “There are learning algorithms which are unstable [13], [25], [179], especially randomized learning approaches such as random vector functional link (RVFL) neural network.”
MultiBoosting: A Technique for Combining Boosting and Wagging (2000, methodology): “Previous bias/variance analyses of decision committee performance Breiman (1996b) shows that bagging can be expected to reduce the variance of a classifier.”
SPLIT SELECTION METHODS FOR CLASSIFICATION TREES (1997, background): “Breiman (1996a) noted that CART and neural networks are in this category, while LDA and nearest neighbor methods are stable.”
Multistrategy ensemble learning: reducing error by combining ensemble learning techniques (2004, background): “This measure is claimed to improve prediction accuracy [5], [2].”
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization (2000, methodology): “Many authors have demonstrated signiﬁcant performance improvements through ensemble methods (Breiman, 1996b; Kohavi & Kunz, 1997; Bauer & Kohavi, 1999; Maclin & Opitz, 1997).”
Ensemble deep learning for regression and time series forecasting (2014, methodology): “Ensemble methods, which work on a higher level to improve the performance of “unstable” predictors [9] such as decision tree and neural networks, have been successfully employed for solving pattern classification, regression, time series forecasting and fault prediction problems.”
A Uni(cid:12)ed Bias-Variance Decomposition (background): “De(cid:12)nition 5 (Breiman, 1996a) A learner is order-correct on an example x i(cid:11) 8 y 6 = y (cid”
Understanding Random Forests: From Theory to Practice (2014, background): “The author empirically shows that Random Forests give results that are competitive with boosting [Freund and Schapire, 1995] and arcing algorithms [Breiman, 1996], which both are designed to reduce bias while forests focus on variance reduction.”
Improve Computer-Aided Diagnosis With Machine Learning Techniques Using Undiagnosed Samples (2007, methodology): “Other well-known algorithms in this category include Arc- x4 [6], LogitBoost [16], etc. Ensemble learning has already been successfully applied to computer-aided diagnosis.”
An Introduction to Boosting and Leveraging (2002, background): “Although this algorithm seems somewhat simplistic, it has the nice property that it tends to reduce the variance of the overall estimate f(x) as discussed for regression [27, 79] and classification [26, 75, 51].”
Data Distillation for Controlling Specificity in Dialogue Generation (2017, methodology): “…that uses subsets of the original data to produce a series of models and then ”boosts” their performance by combining them together; and bagging (Breiman, 1996a), which generates additional data for training using the original dataset to produce multisets of the same size as the original…”
Multi-agent query reformulation: Challenges and the role of diversity (2019, unknown): “Ensemble methods are known to reduce this variance (Freund, 1995; Breiman, 1996a;b).”