Nonconcave penalized likelihood with a diverging number of parameters

By Jianqing Fan, Heng Peng. 2004

1,070 citations recorded by the source corpus.

A class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by Fan and Li to simultaneously estimate parameters and select important variables. They demonstrated that this class of procedures has an oracle property when the number of parameters is finite. However, in most model selection problems the number of parameters should be large and grow with the sample size. In this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to ∞ as the sample size increases. Under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators. Furthermore, the consistency of the sandwich formula of the covariance matrix is demonstrated. Nonconcave penalized likelihood ratio statistics are discussed, and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions. The asymptotic results are augmented by a simulation study, and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/0bfd57b42c4620d3f51d0c75a3919eb9c7b73e83

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Regularized k-means clustering of high-dimensional data and its asymptotic consistency (2012, methodology): “Whereas the selection consistency in regression has been obtained in [9, 29], analogous results in the context of cluster analysis seem rare.”
Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “For penalized methods with unbiasedness and selection features, Fan and Peng (2004) proved the existence, variable selection consistency (1.4) and asymptotic estimation efficiency of some local minimizer of the penalized loss under the dimensionality constraint p= o(nr) with r = 1/3,1/4 or 1/5…”
Shrinkage estimation and selection for multiple functional regression (2011, methodology): “Large sample properties of shrinkage estimation with SCAD penalty have been established in the literature (Fan and Li, 2001; Fan and Peng, 2004; Wang et al., 2008).”
Panel Data Models with Grouped Factor Structure Under Unknown Group Membership (2013, methodology): “For further discussion of the shrinkage methods, we refer to Tibshirani (1996), Fan and Li (2001), Fan and Peng (2004), Zou (2006), Cheng et al. (2013), Lu and Su (2013), and references therein.”
Tuning Parameter Selection for the Adaptive Lasso Using ERIC (2015, background): “This is often referred to literature as a diverging number of parameters setting, after Fan and Peng (2004).”
Forward Regression for Ultra-High Dimensional Variable Screening (2009, methodology): “Intuitively, if some of the nonzero coefficients converge to zero too fast, they cannot be identified consistently; see Fan and Peng (2004).”
A Selective Review of Group Selection in High-Dimensional Models. (2012, background): “…penalty with ρ(x; λ,γ) = λ|x|γ ,0< γ ≤ 1 (Frank and Friedman, 1993); (b) the SCAD penalty with ρ(x;λ,γ) = λ ∫ |x| 0 min{1, (γ− t/λ)+/(γ−1)}dt, γ > 2 (Fan and Li, 2001; Fan and Peng, 2004), where for any a ∈R, a+ denotes its positive part, that is, a+ = a1{a≥0}; (c) the minimax concave penalty…”
Adaptive penalized quantile regression for high dimensional data (2013, methodology): “Various methods have been developed to identify the unknown model and estimate the corresponding coefficients simultaneously for the high dimensional sparse model (see Fan and Peng, 2004; Huang et al., 2008a, 2008b), which mostly focus on the penalized least squares regression.”
A Simple Two-Sample Test in High Dimensions Based on L2-Norm (2020, background): “In regression, feature screening and selection, and dimension reduction are the mainstream approaches (Cheng and Wu 2013; Cook and Li 2002; Fan and Li 2001; Fan and Lv 2008; Fan and Peng 2004; Fan and Song 2010; Lv and Fan 2009; Li et al. 2012; Tibshirani 1996; Xia 2007, 2008).”
GEE analysis of clustered binary data with diverging number of covariates (2011, background): “It is weaker than the condition n − 1 p 5 n = o (1) in Fan and Peng (2004) and Lam and Fan (2008) for asymptotic normality.”
Discussion: One-step sparse estimates in nonconcave penalized likelihood models (2008, background): “In the theoretical front, Fan and Peng [4] proved that the SCAD has the oracle property when the number of variables is a certain fraction power of the sample size.”
A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, background): “Fan and Li (2001) study the oracle properties of nonconcave penalized likelihood estimators in the ﬁnite-dimensional setting, and Fan and Peng (2004) extend their results to the moderate dimensional setting with p = o ( n 1 / 5 ) or o ( n 1 / 3 ).”
NONCONCAVE PENALIZED M-ESTIMATION WITH A DIVERGING NUMBER OF PARAMETERS (2011, methodology): “Thus, in fact, we show that the nonconcave penalized M-estimators have the so-called oracle property in the sense discussed by Fan and Li (2001) and Fan and Peng (2004).”
Semiparametric Ultra-High Dimensional Model Averaging of Nonlinear Dynamic Time Series (2018, methodology): “The condition A8 is a commonly used condition on the penalty function pλ(·), similar to that in Fan and Peng (2004).”
Best subset selection, persistence in high-dimensional statistical learning and optimization under l1 constraint (2006, methodology): “A study of regularization using general penalizations was conducted by Fan and Li [9] and by Fan and Peng [10].”