Variable Selection in Semiparametric Regression Modeling.

By Runze Li, Hua Liang. Annals of Statistics, 2008

317 citations recorded by the source corpus.

In this paper, we are concerned with how to select significant variables in semiparametric modeling. Variable selection for semiparametric regression models consists of two components: model selection for nonparametric components and select significant variables for parametric portion. Thus, it is much more challenging than that for parametric models such as linear models and generalized linear models because traditional variable selection procedures including stepwise regression and the best subset selection require model selection to nonparametric components for each submodel. This leads to very heavy computational burden. In this paper, we propose a class of variable selection procedures for semiparametric regression models using nonconcave penalized likelihood. The newly proposed procedures are distinguished from the traditional ones in that they delete insignificant variables and estimate the coefficients of significant variables simultaneously. This allows us to establish the sampling properties of the resulting estimate. We first establish the rate of convergence of the resulting estimate. With proper choices of penalty functions and regularization parameters, we then establish the asymptotic normality of the resulting estimate, and further demonstrate that the proposed procedures perform as well as an oracle procedure. Semiparametric generalized likelihood ratio test is proposed to select significant variables in the nonparametric component. We investigate the asymptotic behavior of the proposed test and demonstrate its limiting null distribution follows a chi-squared distribution, which is independent of the nuisance parameters. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed variable selection procedures.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/716f79eeacb728825875cacb0530744d5d898f62

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Yingcun Xia
2. Chenlei Leng
3. Heng Lian
4. Hansheng Wang
5. Hua Liang
6. Jianqing Fan
7. Byeong U. Park
8. Tao Hu

Citations in the corpus

• Journal of the American Statistical Association Linear or Nonlinear? Automatic Structure Discovery for Partially Linear Models Linear or Nonlinear? Automatic Structure Discovery for Partially Linear Models (unknown): “…Hong 1991; Green and Silverman 1994; Liang, Hardle, and Carroll 1999), penalized regression splines (Ruppert, Wand, and Carroll 2003; Liang 2006; Wang, Li, and Huang 2008), kernel smoothing (Speckman 1988), and local polynomial regression (Fan and Gijbels 1996; Fan and Li 2004; Li and Liang 2008).”
• Simultaneous variable selection and estimation in semiparametric modeling of longitudinal/clustered data (2013, background): “In the past two decades, there has been a considerable amount of research to study additive, partially linear models (APLM); see Opsomer and Ruppert [27], Härdle, Liang and Gao [12], Li [15], Fan and Li [9], Liang et al. [18], Liu, Wang and Liang [21], Ma and Yang [24], among others.”
• Partial linear single index models with distortion measurement errors (2013, background): “Related works include Li and Liang (2008) for the generalized varying-coefficient partially linear model, Liang et al. (2010) for PLSiM and Wang et al. (2011) for the generalized additive partial linear model.”
• GENERALIZED ADDITIVE PARTIAL LINEAR MODELS WITH HIGH-DIMENSIONAL COVARIATES (2013, methodology): “Li and Liang (2008) adopted the Fan and Li (2001) variable selection procedures for parametric models via nonconcave penalized quasi-likelihood, but their models do not cover the GAPLM and they considered only the fixed-dimensional case.”
• Regularization Parameter Selections via Generalized Information Criterion (2010, background): “Recently, more researchers applied penalized approaches to study variable selections (e.g., Park and Hastie 2007; Wang and Leng 2007; Yuan and Lin 2007; Zhang and Lu 2007; Li and Liang 2008).”
• On the distribution of the adaptive LASSO estimator (2008, unknown): “…property for a variety of penalized maximum likelihood or related estimators (e.g., Bunea (2004), Bunea & McKeague (2005), Fan & Li (2002, 2004), Li & Liang (2007), Wang & Leng (2007), Wang, G. Li and Jiang (2007), Wang, G. Li and Tsai (2007), Wang, R. Li and Tsai (2007), Yuan & Lin (2007),…”
• Shrinkage Estimation of the Varying Coefficient Model (2009, background): “Recently, Li and Liang (2008) carefully studied variable selection for partially linear varying coefficient models, where the parametric components are identified via the SCAD (Fan and Li 2001) but the nonparametric components are selected via a generalized likelihood ratio test, instead of a…”
• ADAPTIVE SEMI-VARYING COEFFICIENT MODEL SELECTION (2012, methodology): “Li and Liang (2008) used SCAD to select parametric components via the generalized likelihood ratio test (Fan, Zhang, and Zhang (2001)) under the assumption that the varying and invariant coefficients are known a priori.”
• Statistical Estimation in Generalized Multiparameter Likelihood Models (2009, methodology): “For a generalized varying-coefficient partially linear model, Li and Liang (2008) used a nonconcave penalized likelihood to select significant variables in the parametric component and a generalized likelihood ratio test to select significant variables in the nonparametric component, assuming that…”
• Mixture of Regression Models With Varying Mixing Proportions: A Semiparametric Approach (2012, methodology): “…using a one-step estimate in semiparametric modeling, for example, a partially linear model (Hunsberger 1994; Severini and Staniswalis 1994), a generalized partially linear single-index model (Carroll et al. 1997), and a generalized varying-coefficient partially linear model (Li and Liang 2008).”
• Statistical Methods with Varying Coefficient Models. (2008, background): “Li and Liang (2007) studied the variable selection issue with the semivarying coefficient models, and Fan and Huang (2005) studied the inference of parametric part a 2 in the semi-varying modeling using GLR test.”
• Variable selection for high-dimensional generalized varying-coefficient models (2012, methodology): “For generalized varying-coefficient partial linear models, Li and Liang (2008) used penalization to select the significant predictors in the parametric components while the nonparametric components were selected by hypothesis testing.”
• NONCONCAVE PENALIZED M-ESTIMATION WITH A DIVERGING NUMBER OF PARAMETERS (2011, methodology): “Müller and Welsh (2005) proposed the selection of a regression model based on combining a robust penalized criterion and a robust conditional expected prediction loss function that is estimated using a stratified bootstrap.”
• A Mallows-Type Model Averaging Estimator for the Varying-Coefficient Partially Linear Model (2018, background): “There is also room for an extension of the present approach to the generalized varyingcoefficient partially linear model (Li and Liang 2008; Lam and Fan 2008) that permits a discrete response variable, and more versatile link functions for error distributions.”
• Penalized least squares for single index models (2011, methodology): “Li and Liang (2008) investigated the nonconcave penalized method for generalized semiparametric partial linear models.”
• A simple approach for varying-coefficient model selection (2009, methodology): “Li and Liang (2008) studied a novel method in semiparametric variable selection.”
• Robust and efficient variable selection for semiparametric partially linear varying coefficient model based on modal regression (2013, methodology): “The generalized mean square error (GMSE) as defined in Li and Liang (2008) is used to evaluate the performance for parametric part GMSE = (β̂ − β)T E(ZZT )(β̂ − β).”