Least angle and ℓ1 penalized regression: A review

By Tim Hesterberg, Nam-Hee Choi, Lukas Meier, Chris Fraley. 2008

310 citations recorded by the source corpus.

Least Angle Regression is a promising technique for variable selection applications, offering a nice alternative to stepwise regression. It provides an explanation for the similar behavior of LASSO (l1-penalized regression) and forward stagewise regression, and provides a fast imple- mentation of both. The idea has caught on rapidly, and sparked a great deal of research interest. In this paper, we give an overview of Least Angle Regression and the current state of related research.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Interpretable classification models for recidivism prediction (2015, methodology): “L 1 -penalized methods are typically used to create linear models that are sparse (Tibshirani 1996, Hesterberg et al. 2008).”
Methods and Models for Interpretable Linear Classification (2014, background): “The interpretability of transparent models is usually improved by tuning sparsity [107, 120, 29, 50, 89, 19, 59, 47], by ensuring monotonic relationships between certain input variables and the predicted outcome [10, 87, 114, 77], and by restricting coefficients to a small set of integer values [23,”
Multiple Change-Point Estimation With a Total Variation Penalty (2010, methodology): “Actually, the corresponding minimization can be solved by using the LAR/LASSO algorithm described in Efron, Hastie, and Tibshirani (2004) and Hesterberg et al. (2008).”
Estimation of a semiparametric recursive bivariate probit model in the presence of endogeneity (2011, methodology): “More theoretical and empirical research in variable selection for nonparametric models is needed; we refer the reader to Hesterberg et al. (2008) and Ruppert, Wand & Carroll (2003) for an introduction.”
Efficient Bayesian inference of subsurface flow models using nested sampling and sparse polynomial chaos surrogates (2014, background): “This is a major advantage over other sparse recovery algorithms that rely on cross-validation to tune the ‘1 regularization constants requiring multiple runs [53].”
Operator-valued Kernels for Learning from Functional Response Data (2015, methodology): “Many variable selection and shrinkage methods for single response regression are extended to the multiple response data case and several algorithms following the corresponding solution paths are proposed (Turlach et al., 2005; Simila and Tikka, 2007; Hesterberg et al., 2008).”
Using Lasso for Predictor Selection and to Assuage Overfitting: A Method Long Overlooked in Behavioral Sciences (2015, methodology): “The estimate for percentage free/reduced lunch is fairly close between methods because this was the predictor with the largest magnitude, meaning that Lasso shrinkage is minimal in this case (Hesterberg et al., 2008).”
A survey of cross-validation procedures for model selection (2009, background): “Let us finally mention the Lasso (Tibshirani, 1996) and other ℓ1 penalization procedures, which have recently attracted much attention (see for instance Hesterberg et al., 2008).”