An improved GLMNET for l1-regularized logistic regression

By Guo-Xun Yuan, Chia-Hua Ho, Chih-Jen Lin. 2011

244 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

CoCoA: A General Framework for Communication-Efficient Distributed Optimization (2016, methodology): “…(∆ α [ k ] ) as deﬁned in (10) were to be minimized exactly, the resulting steps could be interpreted as block-wise Newton-type steps on each coordinate block k , where the Newton-subproblem is modiﬁed to also contain the L 1 -regularizer (Mahajan et al., 2014; Yuan et al., 2012; Qu et al., 2016).”
A distributed block coordinate descent method for training $l_1$ regularized linear classifiers (2014, methodology): “GLMNET [26, 8] is a sequential coordinate descent method that has been demonstrated to be very promising for sequential solution of l1 regularized problems with logistic loss.”
Large Margin Metric Learning for Multi-Label Prediction (2015, methodology): “To address the combinatorial nature of the label space {0, 1} , Zhang and Schneider (2012) use the overgenerating technique (Finley and Joachims 2008) with the cutting plane method.”
Positive-Unlabeled Learning with Non-Negative Risk Estimator (2017, methodology): “iﬁcation of benchmark datasets, models, and optimition algorithms. Name # Train # Test # Feature ˇ p Model g(x;) Opt. alg. A MNIST [29] 60;000 10;000 784 0:49 6-layer MLP with ReLU Adam [20] epsilon [37] 400;000 100;000 2;000 0:50 6-layer MLP with Softsign Adam [20] 20News [38] 11;314 7;532 61;188 0”
Distributed Coordinate Descent for L1-regularized Logistic Regression (2014, methodology): “So for each H m property (11) is already proved in [16].”
Sparse inverse covariance matrix estimation using quadratic approximation (2011, methodology): “ than O(p2) for sparse datasets. Therefore similar quadratic approximation approaches are also eﬃcient for solving the 6 ℓ1-regularized logistic regression problem as shown by [Friedman et al., 2010, Yuan et al., 2012]. In the sequel, we detail three innovations which make our quadratic approximatio”
Primal-Dual Rates and Certificates (2016, methodology): “For those problems, coordinate descent algorithms on the primal formulation have become the state-of-the-art, as in GLMNET (Friedman et al., 2010) and extensions (Shalev-Shwartz & Tewari, 2011; Yuan et al., 2012; 2010).”
L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework (2015, methodology): “Inspired by GLMNET and [34], the work of [3, 18] introduced the idea of a block-diagonal Hessian upper approximation in the distributed L1 context.”
Distributed Optimization for Non-Strongly Convex Regularizers (2016, methodology): “On the theory side, its analysis [15, 17] is not an explicit convergence rate but only asymptotic, as the quadratic upper bounds are not explicitly controlled for safety as with our σ′.”
Parallel Coordinate Descent Newton Method for Efficient $\ell_1$-Regularized Minimization (2013, methodology): “The stopping condition used in the experiments is similar to the outer stopping condition in Yuan et al. (2012).”
FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning (2014, methodology): “FastXML optimizes (14) using the newGLMNET algorithm [37] as implemented in the Liblinear package [14].”
Distributed coordinate descent for generalized linear models with regularization (2016, methodology): “Widely used algorithms that fall into this family are: BBR [1], GLMNET [13], newGLMNET [14].”
A second-order method for strongly convex ℓ1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document}-re (2013, methodology): “,22,26,27,28,29,30] have been developed for the solution of problem (1). Moreover, there has been a series of interesting papers which describe the adaptation of second-order methods to such problems [4,31,8,9,14,15,16,17,21]. The most ecient rstorder methods rely on properties of the ‘ 1-norm to ob”
Proximal Newton-Type Methods for Minimizing Composite Functions (2012, methodology): “λ was chosen to match the value reported in [29], where it was chosen by five-fold cross validation on the training set.”
Recent Advances of Large-Scale Linear Classification (2012, methodology): “An optimization method which can avoid intensive exp/log evaluations is potentially efficient; see more discussion in, for example, [11], [18], [19].”
Blitz: A Principled Meta-Algorithm for Scaling Sparse Optimization (2015, methodology): “GLMNET (Friedman et al., 2010) and LIBLIN- EAR (Yuan et al., 2012), two libraries for 1 -regularized learning, prioritize computation with working set heuristics .”