Bagging, Boosting and the Random Subspace Method for Linear Classifiers

By Marina Skurichina, Robert P. W. Duin. 2002

577 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

In Evaluation & Benchmarks

Citations in the corpus

Deep double descent: where bigger models and more data hurt (2019, background): “Mei & Montanari (2019): This work provides asymptotic analysis for ridge regression over random features Similar double descent behavior, in restricted settings, was investigated in Trunk (1979); Opper (1995; 2001); Skurichina & Duin (2002).”
Towards Open World Recognition (2014, background): “NCM classifiers have a long history of use in vision systems [6] and have multiple extensions, adaptations and applications [7, 39, 44, 18, 24].”
We Can Hear You with Wi-Fi! (2014, unknown): “classifier with very poor performance [44].”
Multiple classifier systems for robust classifier design in adversarial environments (2010, background): “Despite all these works, to our knowledge the effect of randomisation-based techniques on the weight values of linear base classifiers was not considered by any author, even when the behaviour of bagging and the RSM method was specifically investigated for such kinds of classifiers [48,49].”
An empirical Bayes approach to inferring large-scale gene association networks (2005, background): “Regularization of the correlation matrix (e.g. by bagging) reduces this error dramatically (Skurichina and Duin, 2002).”
A Survey on Ensemble Learning for Data Stream Classification (2017, background): “RSM is usually associated with decision trees, however in its general form it can be applied to different base learners, such as nearest neighbors [Ho 1998a] or linear clas-siﬁers [Skurichina and Duin 2002].”
Prototype Classification: Insights from Machine Learning (2009, methodology): “While the analogy between boosting and the SVM has been suggested previously (Skurichina & Duin, 2002), we here establish that the boosting procedure applied on the classical prototype classiﬁer yields the hard margin SVM as a solution when appropriate update rules are chosen: Proposition 5.”
Investigation of the random forest framework for classification of hyperspectral data (2005, background): “…large, random subspace (RS) feature selection can potentially provide improved classiﬁer diversity, while stabilizing parameter estimates, by randomly reducing the number of inputs to each classiﬁer in the ensemble and constructing multiple classiﬁers in the resulting random input space [15], [16].”
A Hybrid Intelligence Approach to Enhance the Prediction Accuracy of Local Scour Depth at Complex Bridge Piers (2020, background): “RS distinct itself from other ensemble technique and become advantageous because it trains the model on randomly selected samples of features opposed to whole feature sets, therefore, reduces correlations between estimators [76].”
Ensemble-based classifiers (2010, methodology): “The idea of using the Dempster–Shafer theory of evidence (Buchanan and Shortliffe 1984) for combining classifiers has been suggested in Shilen (1990) .”
Multi-scale digital terrain analysis and feature selection for digital soil mapping (2010, methodology): “This method is part of various ensemble prediction approaches (Ho, 1998; Bay, 1999; Breiman, 2001; Skurichina and Duin, 2002; Lai et al., 2006).”
Newer Classification and Regression Tree Techniques: Bagging and Random Forests for Ecological Prediction (2006, methodology): “Boosting is competitive with bagging and is used primarily in classifying data with large training sample sizes (Skurichina and Duin 2002).”