A decision-theoretic generalization of on-line learning and an application to boosting

By Yoav Freund, Robert E. Schapire. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Cascade R-CNN: High Quality Object Detection and Instance Segmentation (2019, background): “, the region of largest margin [5], [14].”
PAC-Bayesian Theory Meets Bayesian Inference (2016, neutral): “This paper shows similarities between the early PAC-Bayesian results (McAllester [33], Langford and Seeger [27]), and the Bayesian log-loss bound (Freund and Schapire [11], Kakade and Ng [24]).”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “Combining the outputs of many models is a foundational technique for improving the accuracy and robustness of machine learning models (Dietterich, 2000; Bauer and Kohavi, 1999; Breiman, 1996; Friedman et al., 2001; Lakshminarayanan et al., 2017; Freund and Schapire, 1997).”
OpenSpiel: A Framework for Reinforcement Learning in Games (2019, methodology): “The resulting update reduces to the well-known multiplicative weights update algorithm or Hedge [20], which minimizes regret.”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “If the model is fitted using boosting methods (Freund and Schapire, 1997; Friedman, 2001, 2002), we learn a tree on each single feature in each iteration and then aggregate them together (Lou et al.”
Boosting and Differential Privacy (2010, methodology): “Our algorithm is based on a variant of Freund and Schapire’s Adaboost algorithm [13], due to Schapire and Singer [24].”
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer (2019, methodology): “Multiplicative weight update ( MWU ) technique lies at the core of many learning algorithms [8], [19], [45], [20], [34].”
Iterative random forests to discover predictive and stable high-order interactions (2017, neutral): “5 Relation to AdaBoost In his original paper on RF (Breiman, 2001), Breiman conjectured that in the later stages of iteration, AdaBoost (Freund and Schapire, 1995) emulates RF.”
Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model (2015, neutral): “A P] 5 N ov 2 01 5 The Annals of Applied Statistics 2015, Vol. 9, No. 3, 1350–1371 DOI: 10.1214/15-AOAS848 c© Institute of Mathematical Statistics, 2015”
Interpretable classification models for recidivism prediction (2015, methodology): “we-Taylor, 2000; Berk and Bleich, 2014). Stochastic Gradient Boosting: A popular black-box method that create prediction models in the form of an ensemble of weaker prediction modelsFriedman (2001); Freund and Schapire (1997). 5.1.4. Details on Experimental Design, Parameter Tuning, and Computation ”
BoostClean: Automated Error Detection and Repair for Machine Learning (2017, methodology): “Although there are many possible algorithms [18], we use a powerful technique called Boosting [20] that composes a set of weak learners into a strong learner.”
Mathematical optimization in classification and regression trees (2021, background): “Gradient Boosting Machines (GBMs) is also an iterative procedure based on combining weak tree learners but usually deeper than in AdaBoost.”
Learning customized and optimized lists of rules with mathematical programming (2018, methodology): “Algorithms we compare with include boosted decision trees—AdaBoost [22], logistic regression—LR, support vector machines with RBF kernels—SVM [67], random forests—RF [10], CART, and C5.”
AdaNet: Adaptive Structural Learning of Artificial Neural Networks (2016, methodology): “Φ may, for instance, be the exponential function, Φ( x ) = e x as in the AdaBoost of Freund and Schapire [1997] or the logistic function, Φ( x ) = log(1 + e x ) as in logistic regression.”
A Reductions Approach to Fair Classification (2018, methodology): “The proof follows immediately from the analysis of Freund & Schapire (1996) applied to the Exponentiated Gradient (EG) algorithm (Kivinen & Warmuth, 1997), which in our speciﬁc case is also equivalent to Hedge (Freund & Schapire, 1997).”
BART: Bayesian Additive Regression Trees (2008, background): “These include boosting (Freund & Schapire (1997), Friedman (2001)), random forests (Breiman 2001) and bagging (Breiman 1996), each of which use diﬀerent techniques to ﬁt a linear combination of trees.”