The Strength of Weak Learnability

By Robert E. Schapire. 1989

Subfields: Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

On Network Design Spaces for Visual Recognition (2019, background): “While statistical learning theory [30, 33] has introduced theoretical notions of complexity of machine learning models, these are often not predictive of neural network behavior [36, 37].”
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks (2016, neutral): “First, ensembles usually perform better than any individual networks therein – this has been repeatedly shown in previous studies [3, 19, 4, 10].”
Deep learning in neural networks: An overview (2014, methodology): “Compare earlier, more sophisticated ensemble methods (Schapire, 1990), the contest-winning ensemble Bayes-NN (Neal, 2006) of Section 5.14, and recent related work (Shao, Wu, & Li, 2014).”
Recommendations as Treatments: Debiasing Learning and Evaluation (2016, positive): “Empirical Risk Minimization underlies many successful learning algorithms like SVMs (Cortes & Vapnik, 1995), Boosting (Schapire, 1990), and Deep Networks (Bengio, 2009).”
Learning in the presence of malicious errors (1988, background): “It is worth mentioning at this point that Schapire [20] has shown that if a representation class is not polynomially evaluatable, then it is not e ciently learnable in our model.”
Residual Networks are Exponential Ensembles of Relatively Shallow Networks (2016, background): “Top performers in several competitions use this technique almost as an afterthought [13, 6, 18], but it is rarely thought of as an explicit part of the model structure.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “Ensembles [218, 28] of GPUbased MPCNNs [40, 41] achieved dramatic improvements of long-standing benchmark records, e.”
Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “The LASSO is easy to compute [Osborne, Presnell and Turlach (2000a, 2000b) and Efron et al. (2004)] and has the interpretation as boosting [Schapire (1990), Freund and Schapire (1996) and Friedman, Hastie and Tibshirani (2000)].”
Characterizing the sample complexity of private learners (2013, background): “A line of research (started in [18]) that is very relevant to our paper is boosting learning algorithms, that is, taking learning algorithms that have a big classification error and producing a learning algorithm with small error.”
Evolutionary ensembles with negative correlation learning (2000, methodology): “Since the early 1990's, algorithms based on similar ideas have been developed in many di erent but related forms, such as NN ensembles [5, 2], mixtures of experts [6, 7, 8, 9], and various boosting and bagging methods [10, 11, 12].”
A note on margin-based loss functions in classification (2004, unknown): “See Schapire (1990), Freund (1995), and Freund and Schapire (1996).”
Boosting the margin: A new explanation for the effectiveness of voting methods (1997, unknown): “[17, 20, 34] 19”
Hyperspectral and LiDAR Data Fusion: Outcome of the 2013 GRSS Data Fusion Contest (2014, background): “As indicated by various studies (e.g., [51]–[56]), Random Forests have emerged as a powerful approach with performance similar to boosting [57], [58], SVMs [16], [59], [60], and Neural Networks [61]–[63].”
A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects (2022, methodology): “The boosting algorithm was ﬁrst discussed in a 1990 paper by Schapire [43] in response to a question asked by Kearns and Valiant [68] if a set of weak learners could produce a single strong learner.”
Online Ensemble Learning (2000, methodology): “Schapire’s original boosting algorithm (Schapire, 1990) is a recursive algorithm.”
SHIFTX2: significantly improved protein chemical shift prediction (2011, methodology): “In contrast, Boosting trains subsequent base learners on the mistakes of the previous base learner.”