Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap

By A. Chatterjee, Soumendra Nath Lahiri. 2013

97 citations recorded by the source corpus.

Zou [J. Amer. Statist. Assoc. 101 (2006) 1418-1429] proposed the Adaptive LASSO (ALASSO) method for simultaneous variable selection and estimation of the regression parameters, and established its oracle property. In this paper, we investigate the rate of convergence of the ALASSO estimator to the oracle distribution when the dimension of the regression parameters may grow to infinity with the sample size. It is shown that the rate critically depends on the choices of the penalty parameter and the initial estimator, among other factors, and that confidence intervals (CIs) based on the oracle limit law often have poor coverage accuracy. As an alternative, we consider the residual bootstrap method for the ALASSO estimators that has been recently shown to be consistent; cf. Chatterjee and Lahiri [J. Amer. Statist. Assoc. 106 (2011a) 608-625]. We show that the bootstrap applied to a suitable studentized version of the ALASSO estimator achieves second-order correctness, even when the dimension of the regression parameters is unbounded. Results from a moderately large simulation study show marked improvement in coverage accuracy for the bootstrap CIs over the oracle based CIs.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8383d71822437fe8cb39eadaddaff90430756e6a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Bootstrap Methods in Econometrics (2018, unknown): “Chatterjee and Lahiri (2013) also give conditions under which the bootstrap estimates”
Loss Minimization and Parameter Estimation with Heavy Tails (2013, background): “Chatterjee and Lahiri (2013) study a two-phase sparse estimator for mean-zero noise termed the Adaptive Lasso, proposed in Zou (2006), and show asymptotic convergence results under mild moment assumptions on the noise.”
Confidence regions for high-dimensional generalized linear models under sparsity (2016, background): “Further works on inference in high-dimensional settings include Chatterjee and Lahiri (2013), van de Geer (2014), van de Geer and Stucky (2016), Nickl and van de Geer (2012) and other.”
Confidence intervals for high-dimensional inverse covariance estimation (2014, background): “We note that existing work on statistical inference in high dimensional settings has mostly focused on inference for parameters in linear models and generalized linear models [15, 17, 8, 6, 3, 2, 10].”
Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores (2017, unknown): “Moreover, in the context of linear regression with p n Bradic (2013), Chatterjee and Lahiri (2013), Liu and Yu (2013), Chernozhukov et al. (2013) and Lopes (2014) study properties of various bootstrap methods.”
Bootstrapping and sample splitting for high-dimensional, assumption-lean inference (2016, methodology): “[19] Arindam Chatterjee and Soumendra N.”
Model confidence bounds for variable selection (2016, background): “Similarly, residual bootstrap is able to produce a valid sampling distribution for SCAD, MCP and other studentized estimators Chatterjee and Lahiri (2013).”
High-dimensional simultaneous inference with the bootstrap (2016, methodology): “Bootstrapping the adaptive Lasso in high-dimensional linear models has been put forward and analyzed by Chatterjee and Lahiri (2011, 2013).”
Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-dimensional Linear Regression (2013, background): “…assumption, meaning that the regression coefficients outside an ideal model are small but not necessarily zero (the sum of their absolute values is of the order O(sλn/n)), and imposed the sparse Riesz condition to prove the rate-consistent in terms of the sparsity, bias and the norm of…”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, methodology): “The residual bootstrap method [14] cannot be used in a straightforward way for logistic regression.”
INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS (2018, unknown): “…a turning parameter, and γ ą 0.7 We follow Chatterjee and Lahiri (2011) and use a residual bootstrap method to construct the confidence intervals for the adaptive LASSO estimator; see Chatterjee and Lahiri (2013) for the higher-order refinement of the bootstrap method, and Camponovo (2015) for a…”