A Comparison of Optimization Methods and Software for Large-scale L1-regularized Linear Classification

By Guo-Xun Yuan, Kai-Wei Chang, Cho-Jui Hsieh, Chih-Jen Lin. 2010

261 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

CoCoA: A General Framework for Communication-Efficient Distributed Optimization (2016, methodology): “For those problems, coordinate descent methods on the primal have become state- of-the-art, as in glmnet (Friedman et al., 2010) and extensions (Yuan et al., 2012); see, e.g., the overview in Yuan et al. (2010).”
Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function (2011, background): “For more details we refer the reader to [28] and the references therein; for a survey of recent advances in large-scale linear classification see [29].”
Parallel Coordinate Descent for L1-Regularized Loss Minimization (2011, methodology): “2, theory (Shalev-Shwartz & Tewari, 2009) and extensive empirical results (Yuan et al., 2010) have shown that variants of Shooting are particularly competitive for high-dimensional data.”
Primal-Dual Rates and Certificates (2016, methodology): “For those problems, coordinate descent algorithms on the primal formulation have become the state-of-the-art, as in GLMNET (Friedman et al., 2010) and extensions (Shalev-Shwartz & Tewari, 2011; Yuan et al., 2012; 2010).”
LIBLINEAR: A Library for Large Linear Classification (2008, background): “See the derivation in (Yuan et al., 2010, Appendix B).”
Proximal Methods for Hierarchical Sparse Coding (2010, background): “…(Tibshirani, 1996; Chen et al., 1998; Mallat, 1999; Tropp, 2004, 2006; Wainwright, 2009; Bickel et al., 2009) and the emergence of many efﬁcient algorithmic tools (Efron et al., 2004; Nesterov, 2007; Beck and Teboulle, 2009; Wright et al., 2009; Needell and Tropp, 2009; Yuan et al., 2010).”
L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework (2015, methodology): “Inspired by GLMNET and [34], the work of [3, 18] introduced the idea of a block-diagonal Hessian upper approximation in the distributed L1 context.”
A distributed block coordinate descent method for training $l_1$ regularized linear classifiers (2014, methodology): “Going by the state of the art for sequential solution of such problems (Yuan et al., 2010) we use the coordinatedescent method described in Yuan et al. (2010) for solving (3).”
Communication Efficient Distributed Machine Learning with the Parameter Server (2014, background): “• Only a subset (block) of coordinates is updated in each time: (block) Gauss-Seidel updates are shown to be efficient on sparse data [36, 27].”
Proposal Scaling Distributed Machine Learning with System and Algorithm Co-design (2016, unknown): “Its success is contributed by the following facts: • Taking advantage of the fact that (block) Gauss-Seidel updates are more efficient on sparse data [116, 151], our algorithm only updates a subset (block) of coordinates in each iteration.”
Parallel Coordinate Descent Newton Method for Efficient $\ell_1$-Regularized Minimization (2013, methodology): “Algorithm 1 CDN (Yuan et al., 2010) 1: initialize w0 = 0n×1.”
Sparse inverse covariance matrix estimation using quadratic approximation (2011, methodology): “A similar (so called shrinking) strategy is used in SVM and ℓ 1 -regularized logistic regression problems as mentioned in Yuan et al. [2010].”