Efficient L1 Regularized Logistic Regression

By Su-In Lee, Honglak Lee, Pieter Abbeel, Andrew Y. Ng. AAAI Conference on Artificial Intelligence, 2006

431 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/c8f4436c462d5947c1c908f55e8262eb0bba6db0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yoshua Bengio
2. Dacheng Tao
3. Sergey Levine
4. Trevor Darrell
5. Jianfeng Gao
6. Quoc V. Le
7. Zhe Gan
8. Yi Yang

Citations in the corpus

• Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches (2007, methodology): “We trained Multinomial Logistic Regression classifiers on 11 data sets 8 from the 7 This and higher values produce x = 0 as their solution, following from the first-order optimality conditions of the unconstrained problem.”
• Feature subset selection for logistic regression via mixed integer optimization (2016, methodology): “L1-regularized logistic regression [21, 25] can also be used to select feature subsets.”
• 1 Trend Filtering (2009, methodology): “In statistics, the idea of 1 regularization is used in the well-known Lasso algorithm [93] for 1-regularized linear regression, its extensions such as the fused Lasso [94], the elastic net [107], the group Lasso [105], and the monotone Lasso [51], and 1-regularized logistic regression [61, 62, 77].”
• Large-scale sparse logistic regression (2009, methodology): “the iteratively reweighted least squares (IRLS-LARS) [ 18 ].”
• Conditional random fields for activity recognition (2007, background): “[58] consider the specialized case of logistic regression.”
• A Comparison of Optimization Methods and Software for Large-scale L1-regularized Linear Classification (2010, methodology): “Lee et al. (2006) applied the LARS algorithm described in Efron et al. (2004) to find a Newton direction at each step and then used a backtracking line search to minimize the objective value.”
• Enumerate Lasso Solutions for Feature Selection (2017, methodology): “The proposed algorithm can be easily extended to more general sparsity-inducing convex models such as Lasso Cox regression (Tibshirani 1996), Lasso logistic regression (Lee et al. 2006), elastic-net (Zou and Hastie 2005), fusedLasso (Tibshirani et al.”
• An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression (2007, methodology): “Other recently developed computational methods for `1-regularized logistic regression include the IRLS method (Lee et al., 2006; Lokhorst, 1999), a generalized LASSO method (Roth, 2004) that extends the LASSO method proposed in Osborne et al.”
• Comparing Random Forest with Logistic Regression for Predicting Class-Imbalanced Civil War Onset Data (2016, methodology): “If the L 1 regularization is enforced by an L 1 norm constraint on the parameters, then the optimization problem becomes an optimization problem with the number of constraints equal to twice the number of parameters to be learned (Lee et al. 2006).”
• Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. (2012, methodology): “Indeed it has been used successfully in learning logistic regression (Lee et al. 2006b) and in various graphical models (Lee et al. 2006a, Wainwright et al. 2007), as well as in point process models of neuronal spike trains (Kelly et al. 2010).”
• A second-order method for strongly convex ℓ1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _1$$\end{document}-re (2013, background): “Moreover, there has been a series of interesting papers which describe the adaptation of second-order methods to such problems [4,31,8,9,14,15,16,17,21].”
• Scalable training of L1-regularized log-linear models (2007, methodology): “Lee et al. (2006) propose the algorithm irls-lars, inspired by Newton’s method, which iteratively minimizes the function’s second order Taylor expansion, subject to linear constraints.”
• Learning Graphical Model Structure Using L1-Regularization Paths (2007, methodology): “We used L-BFGS to scale the gradient, and found this to be more efficient than the ‘IRLS-LARS’ algorithm of ( LLAN06 ).”
• Association for Computational Linguistics " Why Should I Trust You? " Explaining the Predictions of Any Classifier (2016, unknown): “One popular way to address the opacity of the models is to use alternate models that are more interpretable [5, 10, 11, 20] and provide insight into why predictions were made; unfortunately they do so at the cost of accuracy and flexibility.”