The Smooth-Lasso and other ℓ1+ℓ2-penalized methods

By Mohamed Hebiri, Sara A. van de Geer. 2011

113 citations recorded by the source corpus.

We consider a linear regression problem in a high dimensional setting where the number of covariates p can be much larger than the sample size n . In such a situation, one often assumes sparsity of the regression vector, i.e., the regression vector contains many zero components. We propose a Lasso-type estimator ˆ β Quad (where ‘ Quad ’ stands for quadratic) which is based on two penalty terms. The ﬁrst one is the ℓ 1 norm of the regression coeﬃcients used to exploit the sparsity of the regression as done by the Lasso estimator, whereas the second is a quadratic penalty term introduced to capture some additional information on the setting of the problem. We detail two special cases: the Elastic-Net ˆ β EN introduced in [42], which deals with sparse problems where correlations between variables may exist; and the Smooth-Lasso 1 ˆ β SL , which responds to sparse problems where successive regression coeﬃcients are known to vary slowly (in some situations, this can also be interpreted in terms of correlations between successive variables). From a theoretical point of view, we establish variable selection consistency results and show that ˆ β Quad achieves a Sparsity Inequality, i.e., a bound in terms of the number of non-zero components of the ‘true’ regression vector. These results are provided under a weaker assumption on the Gram matrix than the one used by the Lasso. In some situations this guarantees a signiﬁcant improvement over the Lasso. Furthermore, a simulation study is conducted and shows that the S-Lasso ˆ β SL performs better than known methods as the Lasso, the Elastic-Net ˆ β EN , and the Fused-Lasso (introduced in [30]) with respect to the estimation accuracy. This is especially the case when the regression vector is ‘smooth’, i.e., when the variations between successive coeﬃcients of the unknown parameter of the regression are small. The study also reveals that the theoretical calibration of the tuning parameters and the one based on 10 fold cross validation imply two S-Lasso solutions with close performance.

Subfields: Computational Mechanics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/276c63583ee19df3dbf82709894f16131e144850

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Weakly decomposable regularization penalties and structured sparsity (2012, methodology): “[2009], Jacob et al. [2009], Jenatton et al. [2011] and Micchelli et al. [2010]. We will provide sharp versions of the oracle inequalities given in Bach [2010] and extend the sharp oracle results for the Lasso and nuclear norm penalization as given in Koltchinskii et al.”