Trust region Newton methods for large-scale logistic regression

By Chih-Jen Lin, Ruby Chiu-Hsing Weng, S. Sathiya Keerthi, Alex Smola. International Conference on Machine Learning, 2007

333 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a8bec570d0e649e58eaafb927378f88c4b023a96

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Distributed Newton Methods for Regularized Logistic Regression (2015, methodology): “[11] apply conjugate gradient (CG) methods to solve (8).”
Iteration complexity of feasible descent methods for convex optimization (2014, methodology): “With Theorem 2.8, the method by Chang et al. (2008) has global linear convergence.”
Stochastic Gradient Descent Tricks (2012, methodology): “either the standard SVM solvers SVMLight and SVMPerf [7] or the superlinear optimization algorithm TRON [11] Figure 2 reports results achieved for a linear model trained on the ALPHA task of the 2008 Pascal Large Scale Learning Challenge using the squared hinge loss `(m) = max{0, 1−m}(2).”
A second-order method for strongly convex ℓ1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document}-re (2013, background): “Moreover, there has been a series of interesting papers which describe the adaptation of second-order methods to such problems [4,31,8,9,14,15,16,17,21].”
PASSCoDe: Parallel ASynchronous Stochastic dual Co-ordinate Descent (2015, background): “on, and many others. Problem (1) is usually called the primal problem, and can usually be solved by Stochastic Gradient Descent (SGD) (Zhang, 2004; Shalev-Shwartz et al., 2007), second order methods (Lin et al., 2007), or primal coordinate descent algorithms (Chang et al., 2008; Huang et al., 2009).”
Bundle Methods for Regularized Risk Minimization (2010, methodology): “In the case of logistic regression, we compare BMRM to the state of the art trust region Newton method for logistic regression (Lin et al., 2008) which is also available in the LIBLINEAR package (option “-s 0”).”
A Parallel SGD method with Strong Convergence (2013, methodology): “(b) For convexf , our method can also use other algorithms (e.g., L-BFGS, TRON etc.) as a replacement forsgd to optimizef̂p, leading to interesting possibilities.”
LIBLINEAR: A Library for Large Linear Classification (2008, methodology): “5 In addition to software documentation, theoretical properties of the algorithms and comparisons to other methods are in Lin et al. (2008) and Hsieh et al. (2008).”
Optimized cutting plane algorithm for support vector machines (2008, background): “SVMLin (Sindhwani & Keerthi, 2007; Chapelle, 2007), LibLinear (Lin et al., 2007), SVM (Joachims, 2006) and BMRM (Teo et al.”
Convergence Rate Analysis of a Stochastic Trust-Region Method via Supermartingales (2019, methodology): “With the rise of interest in nonconvex optimization, the ML community started to consider a classical alternative to gradient descent/line search methods - trust region methods [9, 17, 11].”
Optimized Cutting Plane Algorithm for Large-Scale Risk Minimization (2009, methodology): “Binary SVM solvers employing the hinge risk R(w) = 1 m ∑ m i=1 max{0,1−yi〈w,xi〉} especially have been studied extensively (e.g., Joachims, 1999; Zanni et al., 2006; Chang and Lin, 2001; Sindhwani and Keerthi, 2007; Chapelle, 2007; Lin et al., 2007).”
Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation (2007, background): “…a universal reproducing kernel Hilbert space and an empirical version of the above problem is expressed by the following quadratic program. minimize where (cid:13) (cid:13) (cid:3)(cid:3)(cid:4) (cid:1) is an estimate of the importance at the training input points (cid:0) (cid:2)(cid:0)(cid:1) .”