On the Prediction Performance of the Lasso

By Arnak S. Dalalyan, Mohamed Hebiri, Johannes Lederer. 2014

179 citations recorded by the source corpus.

Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of a simple correlation measure into the tuning parameter can lead to a nearly optimal prediction performance of the Lasso even for highly correlated covariates. However, we also reveal that for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. We finally show that our results also lead to near-optimal rates for the least-squares estimator with total variation penalty.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Sparse Partially Linear Additive Models (2014, background): “There is, however, a different class of oracle inequalities, known as “slow rates,” that make no assumptions on the design matrix (Dalalyan, Hebiri, and Lederer 2014).”
Slope meets Lasso: Improved oracle bounds and optimality (2016, background): “d in [13, Theorem 6.1] and in a slightly less general form, with some factor C>1 in front of kXβ − fk2 n in [4]. The numerical constants in Proposition 3.1 are taken from the proof of Theorem 3 in [9] (cf. the inequality preceding (25) in [9]). A notable feature of Proposition 3.1 and of other no”
A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates (2020, background): “Besides, some drawbacks for the true recovering of the not null elements of the β vector are less harmful for the prediction accuracy target (Dalalyan et al., 2017).”
Cross-Validation With Confidence (2017, methodology): “ procedure on the entire data set with the chosen tuning parameter as a nal output. In many important cases, such as bandwidth selection for nonparametric regression [29], and choosing for the Lasso [9], the optimal tuning parameter may depend on the sample size. Therefore, if n te=n!0, then the opt”