Elastic-net regularization in learning theory
By Christine De Mol, Ernesto De Vito, Lorenzo Rosasco. Journal of Complexity, 2008
346 citations recorded by the source corpus.
Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie [H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B, 67(2) (2005) 301-320] for the selection of groups of correlated variables. To investigate the statistical properties of this scheme and in particular its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combinations of elements (features) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular ''elastic-net representation'' of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed in the above-cited work.
Subfields: Statistical and Nonlinear Physics, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/39d76b6c7510ae47bf44f7fec23c4a388ef9bc77
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistical and Nonlinear Physics
In Statistics and Probability
- Ernesto De Vito
- Lorenzo Rosasco
- Xiaonan Hu
- Alessandro Toigo
- Yuetian Luo
- Davide Ferrari
- Y. Li
- Yichen Qin
Citations in the corpus
- Model confidence bounds for variable selection (2016, background): “(2011), elastic net Zou and Hastie (2005); De Mol et al. (2009), Adaptive Lasso Zou (2006); Zhang and Lu (2007); Huang et al. (2008), group LassoYuan and Lin (2006); Meier et al. (2008); Simon et al. (2013), minimax convex penalization (MCP)Jiang et al. (2013), and stability selection Meinshausen an”
- Digital Twin: Values, Challenges and Enablers From a Modeling Perspective (2019, neutral): “[382] C.”
- Learning Sets with Separating Kernels (2012, background): “The next result is known (see for example [25]), but we report a short proof for completeness.”