On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines

By Koby Crammer, Yoram Singer. 2002

2,379 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Quantifying Learning Guarantees for Convex but Inconsistent Surrogates (2018, background): “At the same time it is known that inconsistent surrogates often work well in practice (for example, the Crammer-Singer formulation of multi-class SVM [8], or its generalization structured SVM [20, 22]).”
On Structured Prediction Theory with Calibrated Convex Surrogate Losses (2017, background): “Ávila Pires et al. [2] generalize the theoretical framework developed by Steinwart [39] and present results for the multi-class SVM of Lee et al. [23] (the score vectors are constrained to sum to zero) that can be built for any task loss of interest.”
Two-Stream Convolutional Networks for Action Recognition in Videos (2014, methodology): “We consider two fusion methods: averaging and training a multi-class linear SVM [6] on stacked L2-normalised softmax scores as features.”
Meta-Learning With Differentiable Convex Optimization (2019, methodology): “The Crammer and Singer [5] formulation of the multi-class SVM is:”
Kernel methods in machine learning (2007, neutral): “• The multiclass SVM formulation of [35] can be recovered as a special case for kernels that are diagonal with respect to the outputs, that is, k((x, y), (x′, y′)) = δy,y′k(x,x ′).”
“Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection (2017, methodology): “We used ﬁve baselines: a majority baseline, a regularized logistic regression classiﬁer (LR), a support vector machine classiﬁer (SVM) (Crammer and Singer, 2001), a bi-directional long short-term memory networks model (Bi-LSTMs) (Hochreiter and Schmidhuber, 1997; Graves and Schmidhu-ber, 2005), and…”
Domain Generalization for Object Recognition with Multi-task Autoencoders (2015, methodology): “We evaluated the classiﬁcation accuracies of the learnt features using multi-class SVM with linear kernel (L-SVM) [12].”
A Primer on Neural Network Models for Natural Language Processing (2015, background): “Hinge (multiclass) The hinge loss was extended to the multiclass setting by Crammer and Singer (2002).”
Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization (2013, methodology): “d n+ min ( R2 ; nR2 )!! : As mentioned before, this is better than SGD when 1 ˛n. 27 5.7 Multiclass SVM Next we consider Multiclass SVM using the construction described in Crammer and Singer [5]. Each example consists of an instance vector x i 2Rdand a label y i 2f1;:::;kg. The goal is to learn a”
An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild (2016, methodology): “Two versions of this approach — SynC and SynC — use one-versus-other and Crammer-Singer style [45] loss functions to train classifiers.”
Inferring the Why in Images (2014, methodology): “2While the rest of this baseline uses Crammer and Singer’s multiclass SVM [6], we found a one-vs-rest strategy worked better for the fully automatic baseline (median rank 30 for Crammer and Singer, and median rank 23 for one-vs-rest).”
SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition (2006, methodology): “To implement multiclass SVM in step 3, three variants from the statistics and learning literature have been tried([21], [9], [31]) on small number of samples from our data sets.”
Towards good practice in large-scale learning for image classification (2012, methodology): “We choose the Crammer and Singer formulation, corresponding to L MUL ( x i , y i ; w ) = max y ∆( y i , y ) + w Ty x i − w Ty i x i , which provides a tighter bound on the misclassiﬁcation error [34].”
Ranking with Large Margin Principle: Two Approaches (2002, result): “This favorably compares to the O(l2) required by the recent SYM approach to ordinal regression introduced in [7] or the kl required by the general multi-class approach to SYM [4,8].”