Bagging Predictors

By Leo Breiman. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Maxout Networks (2013, background): “We now note that the representation of piecewise linear functions given in Proposition 4.1 exactly matches a maxout network with two hidden units h 1 ( v ) and h 2 ( v ), with suﬃ-ciently large k to achieve the desired degree of approximation (cid:15) .”
Improving neural networks by preventing co-adaptation of feature detectors (2012, neutral): “A popular alternative to Bayesian model averaging is “bagging” in which different models are trained on different random selections of cases from the training set and all models are given equal weight in the combination [2].”
Analysis of a Random Forests Model (2010, methodology): “This subspace randomization scheme is blended with bagging ([9, 15, 16, 4]) to resample, with replacement, the training data set each time a new individual tree is grown.”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “Combining the outputs of many models is a foundational technique for improving the accuracy and robustness of machine learning models (Dietterich, 2000; Bauer and Kohavi, 1999; Breiman, 1996; Friedman et al., 2001; Lakshminarayanan et al., 2017; Freund and Schapire, 1997).”
QASC: A Dataset for Question Answering via Sentence Composition (2019, methodology): “Drawing inspiration from bootstrapping (Breiman 1996), we create two such datasets with randomly selected distractors from D and use the models fine-tuned on these datasets as mk (i.”
A Deep Reinforcement Learning Chatbot (2017, methodology): “0 dialogues. We split this into a training set with 3255 examples and a test set with 1085 examples. To increase data efﬁciency, we learn an ensemble model through a variant of the bagging technique (Breiman 1996). We create 5 new training sets, which are shufﬂed versions of the original training se”
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (2016, methodology): “4 In the literature on decision tree ensembles, Breiman [8] proposed to use a combination of bagging [7] and random subset selection of features at each node.”
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks (2016, neutral): “First, ensembles usually perform better than any individual networks therein – this has been repeatedly shown in previous studies [3, 19, 4, 10].”
Random Forests (2001, methodology): “The study of error estimates for bagged classiﬁers in Breiman (1996b), gives empirical evidence to show that the out-of-bag estimate is as accurate as using a test set of the same size as the training set.”
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, methodology): “To protect the privacy of training data, this paper improves upon a speciﬁc, structured application of the techniques of knowledge aggregation and transfer (Breiman, 1994), previously explored by Nis-sim et al. (2007), Pathak et al. (2010), and particularly Hamm et al. (2016).”
N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2019, methodology): “Finally, we perform a standard bagging procedure [4] by sampling a subset of time series and a subset of points in x to train each ensemble member.”
Model Fusion via Optimal Transport (2019, methodology): “Ensemble methods [5–7] have long been in use in deep learning and machine learning in general.”
Deep learning in neural networks: An overview (2014, methodology): “, 2011b) are committees (Breiman, 1996; Schapire, 1990; Wolpert, 1992; Hashem and Schmeiser, 1992; Ueda, 2000; Dietterich, 2000a) of GPU-MPCNNs with simple democratic output averaging.”
Learning privately from multiparty data (2016, background): “This result is especially useful in a scenario where there are a large number of parties with weak local classifiers such as a group of connected smart devices with limited computing capability.”
Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware Detection (2017, methodology): “To this end, multiple linear classifiers are learned by sampling uniformly from the training set (a technique known as bagging [14]) and by randomly subsampling the feature set, as suggested by the random subspace method [22].”
Robust multimodal face and fingerprint fusion in the presence of spoofing attacks (2016, methodology): “The Bootstrap AGGregatING (bagging) method [33] is used to add base classifiers to the base ensemble using bootstrap replicates on the training set.”
Security Evaluation of Support Vector Machines in Adversarial Environments (2014, methodology): “using an ensemble technique like bagging to average several classifiers [16].”