On the distribution of the adaptive LASSO estimator

By B. M. Potscher, Ulrike Schneider. 2008

83 citations recorded by the source corpus.

We study the distribution of the adaptive LASSO estimator (Zou (2006)) in finite samples as well as in the large-sample limit. The large-sample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly non-normal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than n^{-1/2} in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the `oracle' property of the adaptive LASSO estimator established in Zou 2006). Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator.

Subfields: Artificial Intelligence, Finance, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/ce514aedbda0e93d94361022ec8da3ad6a9dff3e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Finance

In Statistics and Probability

Citations in the corpus

Bootstrapping Lasso Estimators (2011, background): “…in Theorem 1, it is important to mention the interesting work done by Pötscher and coauthors (Leeb and Pötscher 2006; Leeb and Pötscher 2008; and Pötscher and Schneider 2009), who have studied the (uniform) consistency of estimates of distributions functions of the Lasso, adaptive Lasso, and…”
Confidence sets based on penalized maximum likelihood estimators in Gaussian regression (2008, methodology): “…of Proposition 3: The distribution function FA,n,θ(x) = Pn,θ(n 1/2(θ̂A− θ) ≤ x) of the adaptive LASSO estimator is given by 1(x + n1/2θ ≥ 0)Φ ( −((n1/2θ − x)/2) + √ ((n1/2θ + x)/2)2 + nη2n ) + 1(x + n1/2θ 0)Φ ( −((n1/2θ − x)/2)− √ ((n1/2θ + x)/2)2 + nη2n ) (see Pötscher and Schneider (2009)).”
Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap (2013, background): “…(2006), Wainwright (2006), Bunea, Tsybakov and Wegkamp (2007), Bickel, Ritov and Tsybakov (2009), Huang, Ma and Zhang (2008), Huang, Horowitz and Ma (2008), Zhang and Huang (2008), Meinshausen and Yu (2009), P¨otscher and Schneider (2009), Chatterjee and Lahiri (2011b), Gupta (2012) among others.”
SHRINKAGE ESTIMATION OF REGRESSION MODELS WITH MULTIPLE STRUCTURAL CHANGES (2015, background): “…based on the post Lasso estimate α̂T̂0 As a referee kindly points out, the asymptotic distribution of the post-Lasso estimator is only valid pointwise and it does not provide uniformly valid inference for the regression coefficients; see Pötscher and Leeb (2009) and Pötscher and Schneider (2009).”
Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-dimensional Linear Regression (2013, background): “We should mention that Lasso+mLS suffers the same issue proposed in [44] which studied the distribution of the adaptive Lasso estimator, that is, the finite sample distribution can be highly non-normal when there is not a gap between the decay rate of the nonzero β∗ and the order n−1/2.”
Model Selection (2018, background): “…distribution is typically highly non-normal and that the cdf of PMSEs can not be estimated with reasonable accuracy, also occur with shrinkage estimators or model averaging estimators; cf. Leeb and Pötscher (2006a), Pötscher (2006), Pötscher and Leeb (2007), and Pötscher and Schneider (2007).”
On various confidence intervals post-model-selection (2014, unknown): “…(1963); Dijkstra and Veldkamp (1988); Kabaila (1998, 2009); Kabaila and Leeb (2006); Leeb (2006); Leeb and P¨otscher (2003, 2005, 2006a,b, 2008a,b); Olshen (1973); P¨otscher (1991, 2006); P¨otscher and Leeb (2009); P¨otscher and Schneider (2009, 2010, 2011); Sen (1979); Sen and Saleh (1987).”
The Risk of James–Stein and Lasso Shrinkage (2016, background): “Other articles that have also pointed out limitations concerning inference using Lasso estimation include Pötscher and Leeb (2008, 2009) and Pötscher and Schneider (2009).”
Valid post-selection inference (2013, background): “…by the “Vienna School” of Pötscher, Leeb and Schneider; see, for example, Leeb and Pötscher (2003, 2005, 2006a, 2006b, 2008a, 2008b, 2008c), Pötscher (2006), Leeb (2006), Pötscher and Leeb (2009), Pötscher and Schneider (2009, 2010, 2011), as well as Kabaila and Leeb (2006) and Kabaila (2009).”
CONSISTENT AND CONSERVATIVE MODEL SELECTION WITH THE ADAPTIVE LASSO IN STATIONARY AND NONSTATIONARY AUTOREGRESSIONS (2015, background): “This phenomenon is similar to the one discussed in Pötscher and Schneider (2009).”