Convexity, Classification, and Risk Bounds

By Peter L. Bartlett, Michael I. Jordan, Jon D. McAuliffe. 2006

1,578 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence (2020, methodology): “For machine learning problems using standard unregularized surrogate loss functions (e.g. squared loss for regression, logistic loss for classification), f∗i = 0 (Bartlett et al., 2006).”
Quantifying Learning Guarantees for Convex but Inconsistent Surrogates (2018, background): “Consistency of convex surrogates has been the central question of many studies for such problems as binary classification [2, 24, 19], multi-class classification [23, 21, 1, 17], ranking [11, 4, 5, 18, 15] and, more recently, structured prediction [7, 14].”
On Structured Prediction Theory with Calibrated Convex Surrogate Losses (2017, methodology): “The consistency of convex surrogates is well understood in the case of binary classiﬁcation [46, 4, 39] and there is signiﬁcant progress in the case of multi-class 0-1 loss [45, 42] and general loss functions [2, 35, 44].”
Conditional variance penalties and domain shift robustness (2017, background): “The (truncated) quadratic loss `(Y, fθ(X)) = (Y − fθ(X))(2) is classification-calibrated (Bartlett et al., 2003) and the truncation is even unnecessary in our case.”
Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization (2008, methodology): “(2005) established a one-to-one correspondence between the family of f -divergences and the family of surrogate loss functions (Bartlett et al., 2006), through which the (optimum) “surrogate risk” is equal to the negative of an associated f -divergence.”
Theoretically Principled Trade-off between Robustness and Accuracy (2019, background): “(3) (1)We study the population form of the risk functions, and mention that by incorporating the generalization theory for classification-calibrated losses (Bartlett et al., 2006) one can extend the analysis to finite samples.”
Kernel methods in machine learning (2007, neutral): “[80], Koltchinskii [86], Mendelson [98] and Vapnik [141] for details.”
Breaking the Curse of Dimensionality with Convex Neural Networks (2014, background): “The number of neurons directly depends on the norm γ1, as we now show.”
Entity Resolution and Federated Learning get a Federated Resolution (2018, background): “With the exception of the unhinged loss [27], this would be the case for most proper, classification calibrated and/or nonconvex losses [3, 21, 24].”
Houdini: Fooling Deep Structured Prediction Models (2017, methodology): “The ﬁrst route is to use a consistent differentiable surrogate loss function in place of the task loss [5].”
Functional Gradient Boosting based on Residual Network Perception (2018, background): “Many studies (Zhang, 2004; Steinwart, 2005; Bartlett et al., 2006) have exploited the consistency of convex loss functions for classification problems in terms of the classification error or conditional probability.”
Composite Multiclass Losses (2011, background): “Glasmachers [13] has shown that certain learning algorithms can still behave well when the losses do not satisfy the conditions in these earlier papers because the requirements are actually stronger than needed.”
Robust Truncated Hinge Loss Support Vector Machines (2007, background): “Fisher consistency, also known as classification-calibration ( Bartlett, Jordan, and McAuliffe 2006 ), is a desirable property for a loss function.”