Statistical Analysis of Some Multi-Category Large Margin Classification Methods

By Tong Zhang. 2004

363 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Optimization & Training

Citations in the corpus

Quantifying Learning Guarantees for Convex but Inconsistent Surrogates (2018, background): “Consistency of convex surrogates has been the central question of many studies for such problems as binary classification [2, 24, 19], multi-class classification [23, 21, 1, 17], ranking [11, 4, 5, 18, 15] and, more recently, structured prediction [7, 14].”
On Structured Prediction Theory with Calibrated Convex Surrogate Losses (2017, methodology): “Following Zhang [45], we show that the surrogates of the form (19) Proof.”
Spectrally-normalized margin bounds for neural networks (2017, methodology): “…working in terms of covering numbers seems essential to providing a bound with no explicit dependence on k ; by contrast, prior work primarily (solely?) handles multiclass via a Rademacher complexity analysis on each coordinate of a k -tuple of functions, and pays a factor √ k (Zhang, 2004).”
"On the (Non-)existence of Convex, Calibrated Surrogate Losses for Ranking" (2012, neutral): “The calibration of surrogate losses has been extensively studied for various classiﬁcation settings [1, 26, 27, 18, 19] and for AUC optimization [7, 15].”
Does label smoothing mitigate label noise? (2020, background): “Smoothing is thus consistent for classiﬁcation, i.e., minimising its risk also minimises the classiﬁcation risk [Zhang, 2004a,b, Bartlett et al., 2006].”
Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost (2010, methodology): “Using the widely-used sum-to-zero constraint [8, 7, 17, 11, 16, 19, 18] on the loss function, one can formulate boosting algorithms only for K − 1 classes, by using one class as the base class .”
Why ResNet Works? Residuals Generalize (2019, methodology): “This generalization bound is developed by employing Rademacher complexity which is upper bounded by covering number (see, respectively, [55, 56] and [19, 38]).”
Robust Truncated Hinge Loss Support Vector Machines (2007, background): “E[�( min g(f(X), Y ))|X = x]. Zhang (2004) and Tewari and Bartlett (2005) pointed out the Fisher inconsistency of H1(min g(f(x), y)).”
Multi-class AdaBoost ∗ (2009, background): “Indeed, [22] also argue that with large samples, AdaBoost.”
How to Compare Different Loss Functions and Their Risks (2007, background): “[1], [27], [37], and [36]), and the proofs of these results are not that deep, either.”
Statistical Analysis of Bayes Optimal Subset Ranking (2008, neutral): “Section III introduces the subset ranking problem.”
Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings (2015, background): “…loss functions, have been investigated independently for quite some time (see for example Bartlett et al. (2006), Tewari and Bartlett (2005), Zhang (2004), Steinwart (2007), Reid and Williamson (2011), Calauzènes et al. (2012, 2013), Scott (2012), Duchi et al. (2013), Menon and Williamson…”
Knowing what You Know: valid and validated confidence sets in multiclass and multilabel prediction (2020, background): “Of course, the ideal scenario occurs when the limiting score function s is optimal (cf. Bartlett et al., 2006; Zhang, 2004; Tewari and Bartlett, 2007), so that s ( x, y ) > s ( x, y (cid:48) ) whenever Under this additional condition, the super-oracle conﬁdence set C 1 − α ( X ) in Def.”
Cross-Entropy Loss Functions: Theoretical Analysis and Applications (2023, background): “The concept of Bayes consistency, or the related one of classification calibration (Zhang, 2004a; Bartlett et al., 2006; Steinwart, 2007; Mohri et al., 2018), have been extensively explored in a wide range of contexts, including multi-class classification (Zhang, 2004b; Chen & Sun, 2006; Chen &…”
On the Consistency of Multiclass Classification Methods (2007, methodology): “Corollary 26 in [12] then guarantees there exists a concave function ξ on [0,∞) such that ξ(0) = 0 and ξ(δ) → 0 as δ → 0 and R(f) −R∗ ≤ ξ(RΨ (f) −R∗ Ψ ) .”
Knowing what you know: valid confidence sets in multiclass and multilabel prediction (2020, neutral): “[2, 36, 31]), so that s(x, y) > s(x, y′) whenever P (Y = y | X = x) > P (Y = y′ | X = x).”