A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs

By S. Sathiya Keerthi, Dennis DeCoste. Journal of machine learning research, 2005

363 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/6c66378bcf0c63a25e78c24b544764e7ee073cb5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Mining adversarial patterns via regularized loss minimization (2010, background): “Different settings of loss functions yield different types of classifiers, such as ln(1 + e−yw T x) in logistic regression [11] and max(0, 1 – yw x) as the binary hinge loss in linear support vector machines (SVMs) [12].”
Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss (2015, background): “These datasets are selected to cover different relations between the sample size N = mn and the feature dimensionality d: N ≫ d (Covtype [6]), N ≈ d (RCV1 [25]) and N ≪ d (News20 [22, 23]).”
Minimizing finite sums with the stochastic average gradient (2013, unknown): “, 2004] news 19 996 1 355 191 [Keerthi and DeCoste, 2005] spam 92 189 823 470 [Cormack and Lynam, 2005, Carbonetto, 2009] rcv1Full 697 641 47 236 [Lewis et al.”
A scalable modular convex solver for regularized risk minimization (2007, methodology): “Finally, several papers [26, 10] advocate the use of Newton-like methods to solve Support Vector Machines in the “pri-mal”.”
Efficient algorithms for ranking with SVMs (2010, methodology): “We present in this section the primal training algorithm of a linear SVM classifier along the lines of (Keerthi and DeCoste, 2005; Chapelle, 2007b).”
A dual coordinate descent method for large-scale linear SVM (2008, methodology): “For the convergence, we prove the following theorem using techniques in (Luo & Tseng, 1992): Theorem 1 For L1-SVM and L2-SVM, { α k,i } generated by Algorithm 1 globally converges to an optimal solution α ∗ .”
Feature Weighting for Improved Classifier Robustness (2009, methodology): “This simplifies the optimization procedure considerably [12] Algorithm 3 Training of fscale-LR or fscale-SVM”
Recent Advances of Large-Scale Linear Classification (2012, methodology): “• L2-regularized L2-loss SVM: Existing methods for the primal form include a coordinate descent method [18], a Newton method [24], and a trust region Newton method (LIBLINEAR [7]).”
Cutting-plane training of structural SVMs (2009, methodology): “The L2-SVM-MFN method (Keerthi and DeCoste 2005) avoids explicitly representing N × N matrices using conjugate gradient techniques.”
Sub-Sampled Newton Methods II: Local Convergence Rates (2016, unknown): “Primal optimizations of linear SVMs have already been studied by [28, 15, 34].”
Training linear SVMs in linear time (2006, methodology): “A method that was recently proposed for training linear SVMs is the L2-SVM-MFN algorithm [15].”
Nonlinear optimization and support vector machines (2018, methodology): “Newton-type methods for formulation (16) of L 2 -SVM have been proposed ﬁrst in Mangasarian (2002) and then in Keerthi and DeCoste (2005).”
Bundle Methods for Regularized Risk Minimization (2010, background): “Finally, several papers (Keerthi and DeCoste, 2005; Chapelle, 2007) advocate the use of Newtonlike methods to solve Support Vector Machines in the “primal”.”
Semi-supervised learning for peptide identification from shotgun proteomics datasets (2007, methodology): “The resulting ranked list gives us an unbiased estimate of the q value for each target PSM [6], i.e., of the minimal false discovery rate threshold required to form a set of positive identifications which includes the PSM. Percolator is implemented in C++, using SVM optimization code from SVMlin [2]”
A second-order method for strongly convex ℓ1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document}-re (2013, unknown): “00e-02 [14] news20 19, 996 1, 355, 191 3.”