Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-dimensional Linear Regression

By Hanzhong Liu, Bin Yu. 2013

89 citations recorded by the source corpus.

We study the asymptotic properties of Lasso+mLS and Lasso+Ridge under the sparse high-dimensional linear regression model: Lasso selecting predictors and then modified Least Squares (mLS) or Ridge estimating their coefficients. First, we propose a valid inference procedure for parameter estimation based on parametric residual bootstrap after Lasso+mLS and Lasso+Ridge. Second, we derive the asymptotic unbiasedness of Lasso+mLS and Lasso+Ridge. More specifically, we show that their biases decay at an exponential rate and they can achieve the oracle convergence rate of $s/n$ (where $s$ is the number of nonzero regression coefficients and $n$ is the sample size) for mean squared error (MSE). Third, we show that Lasso+mLS and Lasso+Ridge are asymptotically normal. They have an oracle property in the sense that they can select the true predictors with probability converging to 1 and the estimates of nonzero parameters have the same asymptotic normal distribution that they would have if the zero parameters were known in advance. In fact, our analysis is not limited to adopting Lasso in the selection stage, but is applicable to any other model selection criteria with exponentially decay rates of the probability of selecting wrong models.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/109974f8eee235de94ed270a1cc4bb3deff6d78c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Lasso adjustments of treatment effect estimates in randomized experiments (2015, background): “[25] Liu H, Yu B (2013) Asymptotic properties of Lasso+mLS and Lasso+Ridge in sparse highdimensional linear regression.”
Incremental causal effects (2019, methodology): “mation and inference of one component in a (potentially high-dimensional) linear model. Several approaches have been developed to do inference in highdimensionallinearmodels (Meinshausen et al.,2009; Liu and Yu, 2013;Zhang and Zhang, 2014; Javanmard and Montanari, 2014; Van de Geer et al., 2014; Bel”
Uncertainty Quantification for the Horseshoe (with Discussion) (2016, background): “In contrast, interesting recent work within the context of the sparse linear regression model is directed at obtaining confidence sets that are honest in the full parameter set Zhang and Zhang (2014); van de Geer et al. (2014); Liu and Yu (2013).”
Model confidence bounds for variable selection (2016, background): “Lastly, Liu and Yu (2013) and Chatterjee and Lahiri (2013) have also stated the bootstrap validity for their methods.”
High-dimensional simultaneous inference with the bootstrap (2016, methodology): “A related approach has been proposed by Liu and Yu (2013).”