Random classification noise defeats all convex potential boosters

By Philip M. Long, Rocco A. Servedio. International Conference on Machine Learning, 2008

295 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/6c2818a816146f845e5e9f53a5233295ce904995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Algorithms for Direct 0-1 Loss Optimization in Binary Classification (2013, background): “A margin mi(w) 0 indicates xi is misclassified, while mi(w) ≥ 0 indicates xi is correctly classified and mi represents the “margin of safety” by which the prediction for xi is correct (McAllester, 2007).”
Machine learning \& artificial intelligence in the quantum domain (2017, background): “Often, best choices (the definition of which depends on the context) lead to hard optimization(Long and Servedio, 2010), and some of those can be reduced to QUBOs, but not straightforwardly.”
Minimizing The Misclassification Error Rate Using a Surrogate Convex Loss (2012, background): “A similar result was also essentially shown by Long & Servedio (2008)— they discuss “boosting”, i.e. loss minimization by co-ordinate descent, here we refer more directly to the loss minimizer.”
Explaining AdaBoost (2013, background): “Regarding this last point, Long and Servedio [18] presented an example of a learning problem which shows just how far off a universally consistent algorithm like AdaBoost can be from optimal when this assumption does not hold, even when the noise affecting the data is seemingly very mild.”
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, background): “For example, naive Bayes and random forests are more robust than other common classifiers such as decision trees and support vector machines [35], [36], and that boosting can exacerbate the impact of label noise [37], [38], [39], whereas bagging is a better”
Consistency versus Realizable H-Consistency for Multiclass Classification (2013, background): “Following those authors (and, others, including (Breiman, 2004; Long & Servedio, 2010)), we abstract away estimation error, and study the consequences of minimizing loss with respect to the underlying distribution.”
Robust Loss Functions under Label Noise for Deep Neural Networks (2017, background): “For binary classiﬁcation, it is shown that 0–1 loss is robust to symmetric or uniform label noise while most of the standard convex loss functions are not robust [16, 18].”
Robust Forward Algorithms via PAC-Bayes and Laplace Distributions (2014, methodology): “Experiments show that our algorithms outperform AdaBoost, L1-LogBoost [10], and RobustBoost [11] in a wide range of input noise.”
Learning to Count with CNN Boosting (2016, background): “Indeed, it was shown that some boosting algorithms including AdaBoost are extremely sensitive to outliers [30].”
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences (2019, background): “Although desirable from an optimization standpoint, convex losses have been shown to be prone to outliers [15] as the loss of each individual example unboundedly increases as a function of the activations.”
Learning with Noisy Labels (2013, methodology): “It will be interesting to see if techniques from Computational Convex Analysis [Lucet, 2010] can be used to efficiently compute the biconjugate above.”
Mining Text Data (2012, methodology): “Many have adopted this seminal approach, mostly in its generic version, sometimes using different measures of novelty to select new sentences [91, 101, 65].”
Loss factorization, weakly supervised learning and label noise robustness (2016, background): “[Long and Servedio, 2010] proves that no convex potential(3) is noise tolerant, that is, 0-ALN robust.”
Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach (2016, background): “In the case of low-capacity (typically linear) models, even mild symmetric, i.e. class-independent (versus asymmetric, i.e. class-dependent), label noise can produce solutions that are akin to random guessing [22].”