Sure independence screening in generalized linear models with NP-dimensionality

By Jianqing Fan, Rui Song. Annals of Statistics, 2009

670 citations recorded by the source corpus.

Ultrahigh dimensional variable selection plays an increasingly important role in contemporary scientific discoveries and statistical research. Among others, Fan and Lv (2008) propose an independent screening framework by ranking the marginal correlations. They showed that the correlation ranking procedure possesses a sure independence screening property within the context of the linear model with Gaussian covariates and responses. In this paper, we propose a more general version of the independent learning with ranking the maximum marginal likelihood estimates or the maximum marginal likelihood itself in generalized linear models. We show that the proposed methods, with Fan and Lv (2008) as a very special case, also possess the sure screening property with vanishing false selection rate. The conditions under which that the independence learning possesses a sure screening is surprisingly simple. This justifies the applicability of such a simple method in a wide spectrum. We quantify explicitly the extent to which the dimensionality can be reduced by independence screening, which depends on the interactions of the covariance matrix of covariates and true parameters. Simulation studies are used to illustrate the utility of the proposed approaches. In addition, we � Supported in part by Grant NSF grants DMS-0714554 and DMS-0704337. The bulk of the work was conducted when Rui Song was a postdoctoral research fellow at Princeton University. The authors would like to thank the associate editor and two referees for their constructive comments that improve the presentation and the results of the paper. AMS 2000 subject classifications: Primary 68Q32, 62J12; secondary 62E99, 60F10

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/458dbdfa7ad609b8dfda40d8603fb77f6b4fc53d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Confidence sets for model selection by F -testing (2015, neutral): “In the literature, several screening methods have been proposed with theoretical justi cations (see, e.g., Fan and Lv (2008) and Fan and Song (2010)).”
Linear Hypothesis Testing in Dense High-Dimensional Linear Models (2016, methodology): “We consider the setting of Fan and Song (2010).”
Finding and Leveraging Structure in Learning Problems (2012, methodology): “To address this issue, we propose and study a simple marginal thresholding step to reduce the dimensionality, in the spirit of the diagonal thresholding of [Johnstone and Lu, 2009b] and the SURE screening of [Fan and Song, 2010].”
Model-Free Feature Screening for Ultrahigh Dimensional Data (2011, background): “Furthermore, Fan, Samworth, and Wu (2009) and Fan and Song (2010) extended SIS and ISIS from a linear model to a generalized linear model.”
STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION (2012, methodology): “ition δ 0 using LASSO can be veriﬁed by using the result of Fan and Lv (2011) and the regularity conditions can be veriﬁed by using the concentration inequality of the maximum likelihood estimator in Fan and Song (2010). 9 3.1 Sparse linear regression The ﬁrst example is the canonical problem of the”
Robust rank correlation based screening (2010, methodology): “Fan and Song (2010) extended SIS to ultrahigh dimensional generalized linear models, and Fan, Feng and Song (2011) studied it for ultra-high dimensional additive models.”
Feature Screening via Distance Correlation Learning (2012, methodology): “Compared with the model-based screening procedures (Fan and Lv 2008; Fan, Samworth, and Wu 2009; Wang 2009; Fan and Song 2010; Fan, Feng, and Song 2011), the DCSIS is a model-free screening procedure.”
Journal of the American Statistical Association Estimating False Discovery Proportion under Arbitrary Covariance Dependence Estimating False Discovery Proportion under Arbitrary Covariance Dependence (positive): “Fan and Song 's model was considered by Fan and Song (2010) for high-dimensional variable selection.”
Penalized Wavelets: Embedding Wavelets into Semiparametric Regression (2011, background): “MR1946581 [24] Fan, J. and Song, R. (2010).”
Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data (2013, methodology): “Fan and Song (2010) further extended the methodology to generalized linear models, see also Fan, Samworth and Wu (2009). The problem of nonlinear features screening was addressed in Hall and Miller (2009) using generalized correlation ranking and more systematically in Fan, Feng and Song (2011) usin”
The fused Kolmogorov filter: A nonparametric model-free screening method (2014, methodology): “…nonparametric, have been proposed and studied in recent years [Chang, Tang and Wu (2013), Fan and Fan (2008), Fan, Feng and Song (2011), Fan, Samworth and Wu (2009), Fan and Song (2010), He, Wang and Hong (2013), Li, Zhong and Zhu (2012), Li et al. (2012), Mai and Zou (2013), Zhu et al. (2011)].”
Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models (2016, methodology): “STR 3—Ordering model screening (OMS): we use the maximum marginal likelihood estimator proposed by Fan and Song (2010) to order the covariates.”
Challenges of Big Data Analysis (2013, positive): “We again treat gene DDR1 as response and other genes as predictors, and apply the FGMM instead of Lasso.”
Independent screening for single‐index hazard rate models with ultrahigh dimensional features (2011, methodology): “The covariance condition in the theorem is analogous to that of Fan and Song (2010) for SIS in generalized linear models with Gaussian features.”
High dimensional ordinary least squares projection for screening variables (2015, methodology): “The screening operation has been extended, for example, to generalized linear models (Fan and Fan, 2008; Fan, et al., 2009; Fan and Song, 2010), additive models (Fan, et al., 2011), hazard regression (Zhao and Li, 2012; Gorst-Rasmussen and Scheike, 2013), and to accommodate conditional correlation…”
Conditional quantile screening in ultrahigh-dimensional heterogeneous data (2015, methodology): “Such screening procedures have been studied extensively in various ultrahigh-dimensional contexts, such as generalized linear models (Fan & Song, 2010), additive models (Fan et al., 2011), Cox proportional hazards regression with survival data (Zhao & Li, 2012), and additive hazard models…”
A Simple Two-Sample Test in High Dimensions Based on L2-Norm (2020, background): “In regression, feature screening and selection, and dimension reduction are the mainstream approaches (Cheng and Wu 2013; Cook and Li 2002; Fan and Li 2001; Fan and Lv 2008; Fan and Peng 2004; Fan and Song 2010; Lv and Fan 2009; Li et al. 2012; Tibshirani 1996; Xia 2007, 2008).”
Bayesian Subset Modeling for High-Dimensional Generalized Linear Models (2013, methodology): “Fan and Song (2010) suggested a way to measure the marginal utility for GLMs using marginal regression coefficients or marginal likelihood ratios (with respect to the null model).”