Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches

By Mark W. Schmidt, glenn. fung, Rómer Rosales. 2007

424 citations recorded by the source corpus.

Subfields: Biomedical Engineering, Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Regularizing Text Categorization with Clusters of Words (2016, methodology): “For the learning part we used Matlab and specifically code by Schmidt et al. (2007).”
Learning Efficient Convolutional Networks through Network Slimming (2017, methodology): “An alternative option is to replace the L1 penalty with the smooth-L1 penalty [30] to avoid using sub-gradient at non-smooth point.”
Learning Task Grouping and Overlap in Multi-task Learning (2012, background): “2, which has superlinear convergence (Schmidt et al., 2007; Gafni & Bertsekas, 1984).”
Optimization with Sparsity-Inducing Penalties (2011, neutral): “Surveys and comparisons of algorithms for sparse methods have been proposed by [103] and [130].”
Elements of style (2015, methodology): “There are several techniques that have been developed in the context of L(1)-norm regularized optimization which are applicable to our problem (see [Schmidt et al. 2007; Andrew and Gao 2007] for related reviews and references).”
Unsupervised Template Learning for Fine-Grained Object Recognition (2012, methodology): “A L1 regularization solver [26] is used for optimizing this formula.”
A Unified View of Matrix Factorization Models (2008, background): “One can exploit a variety of approaches for 1-regularized regression (see [33] for survey) in the projection step.”
In2Rec: Influence-based Interpretable Recommendation (2019, neutral): “l=1 |Pl i − αa | (11) The l1 norms |Pl i − αa | in learning Pi can be approximated by the smoothed l1 method [28], which is given by |Pl i − αa | ≈ 1 δ [log(1 + exp(−δ (Pl i − αa ))) + log(1 + exp(δ (Pl i − αa )))] where δ is a parameter to control approximation.”
A Survey of Sparse Representation: Algorithms and Applications (2015, methodology): “Thus, in terms of the different norms used in optimizers, the sparse representation methods can be roughly grouped into five general categories: sparse representation with the l0-norm minimization [36, 37], sparse representation with the lp-norm (0<p<1) minimization [38– 40], sparse representation w”
Gaussian Kullback-Leibler approximate inference (2013, background): “Indeed, the practical utility of such non-differentiable potentials in statistical modelling has driven a lot of research into speeding up algorithms to find the mode of these densities—for example see Schmidt et al. (2007).”
Distributed Sparse Linear Regression (2010, methodology): “The (most favorable) minimum -norm subgradient [30] was utilized for the simulations.”
Graph Regularized Sparse Coding for Image Representation (2011, background): “Several approaches have been proposed to solve the problem of this form [43]–[47].”
Long-Run Productivity Risk: A New Hope for Production-Based Asset Pricing? (2014, positive): “To preserve the smoothness of all variables required to use perturbation methods, the Schmidt et al. (2007) smooth approximation of the absolute value, |x| = lim →+∞ 1 [ ln ( 1 + e− x ) + ln (1 + e x)− 2 ln(2)] , is applied to equation (14), with = 100.557 The moments obtained by employing convex…”
Heterogeneous multitask learning with joint sparsity constraints (2009, neutral): “That is, each task can be either a regression or classification problem, with the inputs lying in a very high-dimensional feature space, but only a small number of the input variables (i.e., predictors) are relevant to each of the output variables (i.e., responses).”
Nonparametric sparsity and regularization (2012, background): “In this context efficient algorithms (see [50, 39] and references therein) as well as theoretical guarantees are now available (see [14] and references therein).”
Large-scale sparse logistic regression (2009, methodology): “An extensive comparison among twelve sparse logistic regression algorithms was given in [ 30 ].”
An improved GLMNET for l1-regularized logistic regression (2011, background): “Many optimization approaches have been proposed; for example, [2, 3, 4, 5, 6].”