Efficiency for Regularization Parameter Selection in Penalized Likelihood Estimation of Misspecified Models

By Cheryl J. Flynn, Clifford M. Hurvich, Jeffrey S. Simonoff. 2013

70 citations recorded by the source corpus.

It has been shown that Akaike information criterion (AIC)-type criteria are asymptotically efficient selectors of the tuning parameter in nonconcave penalized regression methods under the assumption that the population variance is known or that a consistent estimator is available. We relax this assumption to prove that AIC itself is asymptotically efficient and we study its performance in finite samples. In classical regression, it is known that AIC tends to select overly complex models when the dimension of the maximum candidate model is large relative to the sample size. Simulation studies suggest that AIC suffers from the same shortcomings when used in penalized regression. We therefore propose the use of the classical corrected AIC (AIC c ) as an alternative and prove that it maintains the desired asymptotic properties. To broaden our results, we further prove the efficiency of AIC for penalized likelihood methods in the context of generalized linear models with no dispersion parameter. Similar results exist in the literature but only for a restricted set of candidate models. By employing results from the classical literature on maximum-likelihood estimation in misspecified models, we are able to establish this result for a general set of candidate models. We use simulations to assess the performance of AIC and AIC c , as well as that of other selectors, in finite samples for both smoothly clipped absolute deviation (SCAD)-penalized and Lasso regressions and a real data example is considered. Supplementary materials for this article are available online.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/905343bd1793bbf07aef1b405b88843f43a88756

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Guohua Zou
2. Xinyu Zhang
3. Hua Liang
4. Dalei Yu
5. Tomohiro Ando
6. Ker-Chau Li
7. Jimeng Sun
8. Bryan T. Kelly

Citations in the corpus

• Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models (2016, background): “Condition C.4 is used to prove that ‖β̂s − β∗s ‖ converges to zero at certain rate and is similar to Conditions (A.1′ ) and (A.2′) of Flynn, Hurvich, and Simonoff (2013) and Conditions 1 and 2 of Lv and Liu (2014).”
• FLASH: Fast Bayesian Optimization for Data Analytic Pipelines (2016, background): “model misspecification [48, 16, 28] and superior sample complexity compared to more flexible nonparametric techniques [47], we seek parametric models for propagation of error (or other quantitative metrics) in analytic pipelines.”
• Text as Data (2017, background): “In particular, Flynn, Hurvich, and Simonoff (2013) describe a bias-corrected AIC objective for high-dimensional problems that they call AICc.”
• A weight-relaxed model averaging approach for high-dimensional generalized linear models (2017, background): “Similar to Zhang, Li and Tsai (2010) and Flynn et al. (2013), the setting (e) studies the situation where some true predictors are excluded from the dataset.”