Tuning parameter selection in high dimensional penalized likelihood

By Yingying Fan, Cheng Yong Tang. 2013

355 citations recorded by the source corpus.

Determining how to select the tuning parameter appropriately is essential in penalized likelihood methods for high dimensional data analysis. We examine this problem in the setting of penalized likelihood methods for generalized linear models, where the dimensionality of covariates p is allowed to increase exponentially with the sample size n. We propose to select the tuning parameter by optimizing the generalized information criterion with an appropriate model complexity penalty. To ensure that we consistently identify the true model, a range for the model complexity penalty is identified in the generlized information criterion. We find that this model complexity penalty should diverge at the rate of some power of log (p) depending on the tail probability behaviour of the response variables. This reveals that using the Akaike information criterion or Bayes information criterion to select the tuning parameter may not be adequate for consistently identifying the true model. On the basis of our theoretical study, we propose a uniform choice of the model complexity penalty and show that the approach proposed consistently identifies the true model among candidate models with asymptotic probability 1. We justify the performance of the procedure proposed by numerical simulations and a gene expression data analysis.

Subfields: Artificial Intelligence, Molecular Biology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/a50cda55ac39ec5096de1c6d83900f3d0130628d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Tuning Parameter Selection for the Adaptive Lasso Using ERIC (2015, methodology): “43 Recently, several modifications to BIC have been proposed, with the aim been to construct a cri44 terion for selecting the tuning parameter that is selection consistent in high-dimensional settings 45 (e.g., Wang et al., 2009; Wang and Zhu, 2011; Fan and Tang, 2012).”
Cross-Validation With Confidence (2017, methodology): “See [33, 12, 18] for some examples of information criteria based methods that combines regularization parameter tuning and variable selection.”
Predicting the long-term stock market volatility: A GARCH-MIDAS model with variable selection (2020, methodology): “Default spread is the yield spread between BAA and AAA rated bonds, which should affect aggregate volatility, according to Merton (1974). Equity market returns (MKT) in Fama and French (1992) can capture the leverage effect (Black, 1976; Glosten et al.”
Forecasting: theory and practice (2020, background): “rion, such as the Bayesian information criterion (BIC, Schwarz, 1978), to select (or tune) the regularisation parameter governing the degree of shrinkage (often denoted by ). Wang et al. (2009a) and Fan and Tang (2013) modify the BIC penalty to account for a diverging number of features, while Hui e”
From big data analysis to personalized medicine for all: challenges and opportunities (2015, methodology): “There are several approaches to derive “ optimal ” tuning parameters to dictate the number of relevant variables to pursue [79, 80].”
Asymptotically Honest Confidence Regions for High Dimensional Parameters by the Desparsified Conservative Lasso (2014, methodology): “ontanari (2014) using the authors’ code6. The Lasso as well as the conservative Lasso are implemented in R by means of the publicly available glmnetpackage. To choose λn as well as λnode,n, we follow Fan and Tang (2013) and use their Generalized Information Criterion (GIC). In the regression equatio”
Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data (2013, methodology): “A similar BIC criterion was suggested by [28], and a generalized information criterion was considered by [13] for tuning parameter selection in penalized likelihood models.”
Variable selection using pseudo-variables (2018, methodology): “…extensive literature on tuning parameter selection including information-based criteria (Chen and Chen, 2008; Wang et al., 2009; Zhang et al., 2010; Fan and Tang, 2013; Hui et al., 2015), resampling methods (Hall et al., 2009; Meinshausen and Bühlmann, 2010; Feng and Yu, 2013; Sun et al., 2013;…”