A new approach to variable selection in least squares problems

By Michael R. Osborne, Brett Presnell, Berwin A. Turlach. 2000

957 citations recorded by the source corpus.

Subfields: Control and Systems Engineering, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/db31973871f5648cffb02e7590582376e16ad0bb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Fast L1-Minimization Algorithms For Robust Face Recognition (2010, neutral): “The approach was ﬁrst studied in the context of LASSO [49], which inspired a solution to the forward stagewise linear regression problem called LARS [50] and eventually led to the Homotopy algorithms for basis pursuit in [51], [52].”
Variable Selection in Predictive Regressions (2013, methodology): “A more efficient solution can be obtained by using the homotopy algorithm of Osborne, Presnell, and Turlach (2000), which is related to forward stagewise regressions.”
A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems (2011, methodology): “One may use the homotopy/Lars algorithm to compute the complete Lasso path for λ ∈ [0,∞) [13, 28, 29] or simply use a standard convex optimization algorithm to compute the Lasso solution for a finite set of λ.”
Unified LASSO Estimation by Least Squares Approximation (2007, background): “In the ordinary least squares (OLS) setting, Osborne, Presnell, and Turlach (2000) and Efron, Hastie, Johnstone, and Tibshirani (2004) have shown that the solution path of LASSO is piecewise linear and moves in a very predictable manner.”
Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming (2009, neutral): “Hence in order to overcome such an issue, several works [40], [33], [8], [42] have proposed to relax the problem and instead to consider the following convex optimization problem: min β∈R d 1 2 y − Xβ 2 + λβ 1 .”
1 Trend Filtering (2009, methodology): “The QP (13) can be solved by standard convex optimization methods, including general interior-point methods [12, 73, 74, 103] and more specialized methods such as path following [76, 30].”
Joint covariate selection and joint subspace selection for multiple classification problems (2010, background): “For the case of covariate selection we show that it is possible to develop a simple homotopy-based approach that evaluates an entire regularization path efficiently (cf. Efron et al. 2004; Osborne et al. 2000).”
Forward stagewise regression and the monotone lasso (2007, background): “(2004) (the lasso results also in Osborne et al. (2000) and Rosset & Zhu (2003)).”
Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence (2017, methodology): “[63, 64] who proposed an exact homotopy method that solves the LASSO problem for all values of λ.”
A Survey of Sparse Representation: Algorithms and Applications (2015, methodology): “The homotopy method was originally proposed to solve the least square problem with the l1-penalty [94].”
Penalized regression, standard errors, and Bayesian lassos (2010, methodology): “For the computation of the lasso, Osborne et al. (2000a) proposed two algorithms.”
A non-adapted sparse approximation of PDEs with stochastic inputs (2010, methodology): “Numerous solvers based on the active set [54,36], interior-point continuation [21,42] and projected gradient [25,23,39,13,9,62,3,4] methods have been developed for solving the above formulations of the ‘1-minimization problem.”
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization (2007, positive): “memory-ecient. It is also of much interest to investigate the possible adaptation of some of the successful pathfollowing approaches in traditional ‘ 1/cardinality minimization, such as the Homotopy [40] or LARS (least angle regression) [21]. This may be not be completely straightforward, since the ”
Online Learning for Matrix Factorization and Sparse Coding (2009, positive): “This has led us to use instead the LARS-Lasso al-gorithm, a homotopy method (Osborne et al., 2000; Efron et al., 2004) that provides the whole regularization path—that is, the solutions for all possible values of λ .”