On the “degrees of freedom” of the lasso

By Hui Zou, Trevor J. Hastie, Robert Tibshirani. 2007

1,084 citations recorded by the source corpus.

We study the effective degrees of freedom of the lasso in the framework of Stein’s unbiased risk estimation (SURE). We show that the number of nonzero coefficients is an unbiased estimate forthe degrees of freedom of the lasso—a conclusion that requires no special assumption on the predictors. In addition, the unbiased estimator is shown to be asymptotically consistent. With these results on hand, various model selection criteria—Cp, AIC and BIC—are available, which, along with the LARS algorithm, provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least-squares fit.

Subfields: Finance, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Finance

In Statistics and Probability

Citations in the corpus

Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso (2017, neutral): “…for the active set; • again thanks to shrinkage, the effective degrees of freedom of the lasso (at any fixed λ ≥ 0) is equal to the expected number of selected variables (Zou et al., 2007; Tibshirani and Taylor, 2012), whereas the degrees of freedom of both forward stepwise and best subset…”
Estimating treatment effect heterogeneity in randomized program evaluation (2013, neutral): “The general pattern from our simulations is that”
Text as Data (2017, methodology): “The BIC tends to choose simpler models than cross-validation or AICc.”
A Model-Averaging Approach for High-Dimensional Regression (2014, methodology): “These include the lasso method (Tibshirani 1996; Zou, Hastie, and Tibshirani 2007) and its variants (Yuan and Lin 2006; Zou 2006), Bayesian lasso (Park and Casella 2008), least-angle regression (Efron et al. 2004), elastic net (Zou and Hastie 2005), the smoothly clipped absolute deviation penalty…”
Degrees of freedom in lasso problems (2011, neutral): “By Ryan J. Tibshirani and Jonathan Taylor1”
Estimation for High‐Dimensional Linear Mixed‐Effects Models Using ℓ1‐Penalization (2010, methodology): “The use of |{1 ≤ k ≤ p : β̂k 6= 0}| as a measure of the degrees of freedom is motivated by the work of Zou et al. (2007) who show that the expected number of degrees of freedom for the Lasso in a linear model is given by the number of non-zero estimated coefficients.”
Variance estimation in high-dimensional linear models (2014, background): “Reliable estimates of σ2 may be required to compute popular model selection statistics, such as AIC, BIC, or RIC (Akaike, 1974; Foster and George, 1994; Schwarz, 1978; Zou et al., 2007).”
Regularized Multivariate Regression for Identifying Master Predictors with Application to Integrative Genomics Study of Breast Cancer (2008, methodology): “Here, for each lasso regression, the degrees of freedom is estimated by the total number of selected predictors (Zou et al. 2007).”
Functional additive regression (2015, background): “By Yingying Fan1, Gareth M. James and Peter Radchenko2”
Likelihood-Free Inference by Ratio Estimation (2016, background): “Different schemes to choose λ have been proposed that aim at minimising the prediction risk (Zou et al., 2007; Wang and Leng, 2007; Tibshirani and Taylor, 2012; Dutta et al., 2012).”
Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso (2007, methodology): “…cross-validation, generalized cross-validation (Craven and Wahba 1979; Tibshirani 1996; Fan and Li 2001), AIC, AICc, and BIC (Hurvich and Tsai 1989; Zou, Hastie, and Tibshirani 2004) methods can be used to select the optimal regularization parameter λ j after appropriate modiﬁcation, for example,…”