Prediction error of cross-validated Lasso

By Sourav Chatterjee, Jafar Jafarov. 2015

43 citations recorded by the source corpus.

In spite of the wealth of literature on the theoretical properties of the Lasso, there is very little known when the value of the tuning parameter is chosen using the data, even though this is what actually happens in practice. We give a general upper bound on the prediction error of Lasso when the tuning parameter is chosen using a variant of 2-fold cross-validation. No special assumption is made about the structure of the design matrix, and the tuning parameter is allowed to be optimized over an arbitrary data-dependent set of values. The proof is based on a general principle that may extend to other kinds of cross-validation as well as to other penalized regression methods. Based on this result, we propose a new estimate for error variance in high dimensional regression and prove that it has good properties under minimal assumptions.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, positive): “This result tightens the result in Chatterjee and Jafarov [2015] who established a "slow rate" bound (in the context of Lasso) that applies under no assumptions on G.”
Approximate residual balancing: debiased inference of average treatment effects in high dimensions (2016, methodology): “Similar estimators have been considered by Graham et al. [2012, 2016] and Hainmueller [2012] in a setting where exact balancing is possible, with slightly different objection functions. For example, Hainmueller [2012] uses − ∑ i ln(γi) instead of ∑ i γ 2 i , leading to γ = argminγ̃ − ∑ {i:Wi=0} lo”
On cross-validated Lasso in high dimensions (2016, methodology): “Other papers that have been concerned with cross-validation in the context of the Lasso estimator include Chatterjee and Jafarov [19] and Lecué and Mitchell [30].”
Cross-Validation With Confidence (2017, methodology): “[3] study the risk consistency of cross-validated Lasso with a specially designed 2-fold cross-validation variant.”