ON THE ADAPTIVE ELASTIC-NET WITH A DIVERGING NUMBER OF PARAMETERS

By Hui Zou, Hao Helen Zhang. Annals of Statistics, 2009

838 citations recorded by the source corpus.

We consider the problem of model selection and estimation in situations where the number of parameters diverges with the sample size. When the dimension is high, an ideal method should have the oracle property (Fan and Li, 2001; Fan and Peng, 2004) which ensures the optimal large sample performance. Furthermore, the high-dimensionality often induces the collinearity problem which should be properly handled by the ideal method. Many existing variable selection methods fail to achieve both goals simultaneously. In this paper, we propose the adaptive Elastic-Net that combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage. Under weak regularity conditions, we establish the oracle property of the adaptive Elastic-Net. We show by simulations that the adaptive Elastic-Net deals with the collinearity problem better than the other oracle-like methods, thus enjoying much improved finite sample performance.

Subfields: Computational Mechanics, Geometry and Topology

View Paper: https://www.semanticscholar.org/paper/d49fcaf2314659817b433b6fb13e386bc365d034

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Geometry and Topology

Citations in the corpus

Machine Learning Advances for Time Series Forecasting (2020, methodology): “4 The adaLASSO penalty consists in using a weighted 𝓁 1 penalty: where the number of features 𝜔 𝑖 = | 𝛽 | −1 and 𝛽 is the coefficient from the first-step estimation (any consistent estimator of 𝜷 0 ) AdaLASSO can deal with many more variables than observations.”
Complete subset regressions (2013, methodology): “The Elastic Net penalty term includes as special cases the ridge penalty (α = 0) and the Lasso penalty (α = 1). β̂α,ψ is computed using the coordinate descent algorithmdeveloped in Friedman and Tibshirani (2010).”
Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions (2012, methodology): “…the lasso and enhance its stability (selection consistency and asymptotic normality) with high-dimensional data by solving the optimization problem [10]: where the elastic-net estimator ˆ β (enet) in (5) is first computed and then adaptive weights are constructed by: and g is a positive constant.”
Estimating Global Bank Network Connectedness (2017, neutral): “A simple extension of the LASSO, the so-called adaptive elastic net (Zou & Zhang (2009)), not only shrinks and selects, but also has the oracle property, meaning (roughly) that the selected model is consistent for the best Kullback-Liebler approximation to the true DGP.”
High dimensional ordinary least squares projection for screening variables (2015, background): “…Lasso (Zou, 2006; Wang, et al., 2007; Zhang and Lu, 2007), the grouped Lasso (Yuan and Lin, 2006), the LSA estimator (Wang and Leng, 2007), the Dantzig selector (Candes and Tao, 2007), the bridge regression (Huang, et al., 2008), and the elastic net (Zou and Hastie, 2005; Zou and Zhang, 2009).”
Tuning Parameter Selection for the Adaptive Lasso Using ERIC (2015, background): “Note that λ must 162 grow with sample size anyway to achieve selection consistency (Zou, 2006; Zou and Zhang, 2009).”
Feature Screening via Distance Correlation Learning (2012, background): “…clipped absolute deviation (SCAD) (Fan and Li 2001; Kim, Choi, and Oh 2008; Zou and Li 2008), the least angle regression (LARS) algorithm (Efron et al. 2004), the elastic net (Zou and Hastie 2005; Zou and Zhang 2009), the adaptive LASSO (Zou 2006), and the Dantzig selector (Candes and Tao 2007).”
The Smooth-Lasso and other ℓ1+ℓ2-penalized methods (2011, methodology): “In the next section, we introduce the estimator ˆ β Quad deﬁned with the Lasso penalty together with a quadratic penalty.”
THE Mnet method for variable selection (2016, background): “Zou and Zhang (2009) proposed the adaptive Enet estimator and provided sufficient conditions under which it is oracle.”
Forward Regression for Ultra-High Dimensional Variable Screening (2009, methodology): “All those methods are very useful and can be formulated as penalized optimization problems, which could be selection consistent, if the sample size is much larger than the predictor dimension (Fan and Peng, 2004; Huang et al., 2008; Zou and Zhang, 2008).”
Regularized k-means clustering of high-dimensional data and its asymptotic consistency (2012, result): “Whereas the selection consistency in regression has been obtained in [9, 29], analogous results in the context of cluster analysis seem rare.”
The Bayesian elastic net (2010, background): “Zou and Zhang (2009) proposed the adaptive elastic net to achieve this goal.”