Tuning Parameter Selection for the Adaptive Lasso Using ERIC

By Francis K. C. Hui, David I. Warton, Scott D. Foster. 2015

76 citations recorded by the source corpus.

The adaptive Lasso is a commonly applied penalty for variable selection in regression modeling. Like all penalties though, its performance depends critically on the choice of the tuning parameter. One method for choosing the tuning parameter is via information criteria, such as those based on AIC and BIC. However, these criteria were developed for use with unpenalized maximum likelihood estimators, and it is not clear that they take into account the effects of penalization. In this article, we propose the extended regularized information criterion (ERIC) for choosing the tuning parameter in adaptive Lasso regression. ERIC extends the BIC to account for the effect of applying the adaptive Lasso on the bias-variance tradeoff. This leads to a criterion whose penalty for model complexity is itself a function of the tuning parameter. We show the tuning parameter chosen by ERIC is selection consistent when the number of variables grows with sample size, and that this consistency holds in a wider range of contexts compared to using BIC to choose the tuning parameter. Simulation show that ERIC can significantly outperform BIC and other information criteria proposed (for choosing the tuning parameter) in selecting the true model. For ultra high-dimensional data (p > n), we consider a two-stage approach combining sure independence screening with adaptive Lasso regression using ERIC, which is selection consistent and performs strongly in simulation. Supplementary materials for this article are available online.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/f0db6fc31357098c575f8f0f2e66e4b1f49f58a5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Cross-Validation With Confidence (2017, methodology): “See [33, 12, 18] for some examples of information criteria based methods that combines regularization parameter tuning and variable selection.”
Variable selection using pseudo-variables (2018, methodology): “…on tuning parameter selection including information-based criteria (Chen and Chen, 2008; Wang et al., 2009; Zhang et al., 2010; Fan and Tang, 2013; Hui et al., 2015), resampling methods (Hall et al., 2009; Meinshausen and Bühlmann, 2010; Feng and Yu, 2013; Sun et al., 2013; Shah and Samworth,…”