The Lasso, correlated design, and improved oracle inequalities

By Sara A. van de Geer, Johannes Lederer. 2011

56 citations recorded by the source corpus.

We study high-dimensional linear models and the $\ell_1$-penalized least squares estimator, also known as the Lasso estimator. In literature, oracle inequalities have been derived under restricted eigenvalue or compatibility conditions. In this paper, we complement this with entropy conditions which allow one to improve the dual norm bound, and demonstrate how this leads to new oracle inequalities. The new oracle inequalities show that a smaller choice for the tuning parameter and a trade-off between $\ell_1$-norms and small compatibility constants are possible. This implies, in particular for correlated design, improved bounds for the prediction error of the Lasso estimator as compared to the methods based on restricted eigenvalue or compatibility conditions only.

Subfields: Control and Systems Engineering, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6a870a2d9bbbc4067b28ee6105e754e7c3502af0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization (2012, background): “In [25, 53], improvements of the slow rate bound are derived.”
How Correlations Influence Lasso Prediction (2012, methodology): “However, recent results in [26] suggest that this is not true.”
Prediction error of cross-validated Lasso (2015, background): “Other authors, such as Tibshirani [34, 35], Greenshtein and Ritov [14], Hastie et al. [15], Efron et al. [10], Zou et al. [44], van de Geer and Lederer [38] and Friedman [12] recommend using cross-validation to select the value of the tuning parameter.”
Likelihood-Free Inference by Ratio Estimation (2016, background): “As r ( x, θ ) is the ratio between p ( x | θ ) and p ( x ), we can consider the estimate ˆ r ( x, θ ) in (15) to provide an implicit estimate ˆ p ( x | θ ) of the intractable model pdf p ( x | θ ), The estimate is implicit because we have not explicitly estimated the marginal pdf p ( x ).”