Non-Concave Penalized Likelihood with NP-Dimensionality
By Jianqing Fan, Jinchi Lv. 2009
434 citations recorded by the source corpus.
Penalized likelihood methods are fundamental to ultrahigh dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial (NP) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions. This fills a long-standing gap in the literature where the dimensionality is allowed to grow slowly with the sample size. Our results are also applicable to penalized likelihood with the L1-penalty, which is a convex function at the boundary of the class of folded-concave penalty functions under consideration. The coordinate optimization is implemented for finding the solution paths, whose performance is evaluated by a few simulation examples and the real data analysis.
Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Control and Systems Engineering
In Statistics and Probability
Citations in the corpus
- High‐dimensional macroeconomic forecasting and variable selection via penalized regression (2019, background): “Result (c) guarantees that the non-zero coefficients can be efficiently estimated as if A were known in advance; see Fan and Lv (2011) for a complete guide in an i.”
- A Split-and-Conquer Approach for Analysis of Extraordinarily Large Data (2014, methodology): “Following Zhang and Huang (2008) and also to obtain a slightly stronger sign consistency result than that of Fan and Lv (2011) (when K = 1), we introduce two diverging sequences vn,K and un,K that depend on the total sample size n and the number of subsets K, and assume that A2 vn,K =…”
- Tight conditions for consistency of variable selection in the context of high dimensionality (2011, methodology): “on procedures [2, 45] and to marginal regression or correlation screening [18, 42]. Promising results for feature selection are also obtained by conformal prediction [20], (minimax) concave penalties [16, 17, 44], Bayesian approach [37] and higher criticism [15]. Extensions to other settings includi”
- Independent screening for single‐index hazard rate models with ultrahigh dimensional features (2011, background): “The notion of non-polynomial dimensionality (Fan and Lv, 2011) has been conceived to describe such ultrahigh dimensional settings which are formally analysed in an asymptotic regime where p grows at a non-polynomial rate with n.”
- Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data (2013, background): “A similar argument can be found in [8].”
- A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression (2014, background): “Within the class of non-convex penalties, Fan and Lv (2011) and Bradic et al. (2011) established asymptotic normality for regression parameters on the support based on oracle properties and minimal signal strength conditions.”
- Robust rank correlation based screening (2010, background): “…regression [Frank and Friedman (1993), Huang, Horowitz and Ma (2008)], LASSO [Tibshirani (1996), van de Geer (2008)], elastic-net [Zou and Hastie (2005)], adaptive LASSO [Zou (2006)], SCAD [Fan and Li (2001), Fan and Peng (2004), Fan and Lv (2011)] and Dantzig selector [Candes and Tao (2007)].”
- A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations (2015, methodology): “For the nonconvex penalty, under the oracle properties, the asymptotic normality property of the estimators is established by Fan and Lv (2011), which requires strong conditions, such as the minimal signal condition.”
- High dimensional thresholded regression and shrinkage effect (2014, methodology): “…in estimated model, we consider the regularized estimator β on the union of coordinate subspaces S M/ 2 = { β ∈ R p : β 0 < M/ 2 , as exploited in Fan and Lv (2011) to characterize the global optimality of nonconcave penalized likelihood estimators. b Thus throughout the paper, the regularized…”
- Tuning parameter selection in high dimensional penalized likelihood (2013, methodology): “2) were made in Fan and Song (2010) and Bühlmann and van de Geer (2011) for studying high dimensional penalized likelihood methods. Since our interest is in tuning parameter selection, we impose condition 4 to ensure that the true model can be recovered by regularization methods. No requirements in ”
- High-Dimensional Sparse Additive Hazards Regression (2012, methodology): “In addition to their classical applications for survival analysis in public health, survival models have been widely used to model time-to-event data for credit risk analysis in finance and economics (Jarrow 2009; Fan, Lv, and Qi 2011).”
- Estimation and Selection of Spatial Weight Matrix in a Spatial Lag Model (2019, methodology): “The BCD algorithm is closely related to the iterative coordinate descent of Fan and Lv (2011), and is also discussed by Friedman., Hastie, and Tibshirani (2010) and Dicker, Huang, and Lin (2013).”