Sure independence screening for ultrahigh dimensional feature space

By Jianqing Fan, Jinchi Lv. 2006

2,807 citations recorded by the source corpus.

Summary. Variable selection plays an important role in high dimensional statistical modelling which nowadays appears in many areas and is key to various scientific discoveries. For problems of large scale or dimensionality p, accuracy of estimation and computational cost are two top concerns. Recently, Candes and Tao have proposed the Dantzig selector using L1‐regularization and showed that it achieves the ideal risk up to a logarithmic factor log (p). Their innovative procedure and remarkable result are challenged when the dimensionality is ultrahigh as the factor log (p) can be large and their uniform uncertainty principle can fail. Motivated by these concerns, we introduce the concept of sure screening and propose a sure screening method that is based on correlation learning, called sure independence screening, to reduce dimensionality from high to a moderate scale that is below the sample size. In a fairly general asymptotic framework, correlation learning is shown to have the sure screening property for even exponentially growing dimensionality. As a methodological extension, iterative sure independence screening is also proposed to enhance its finite sample performance. With dimension reduced accurately from high to below sample size, variable selection can be improved on both speed and accuracy, and can then be accomplished by a well‐developed method such as smoothly clipped absolute deviation, the Dantzig selector, lasso or adaptive lasso. The connections between these penalized least squares methods are also elucidated.

Subfields: Control and Systems Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

Stability selection (2008, methodology): “The iterative sure independence screening procedure (Fan and Lv, 2008) entails OMP as a special case.”
Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “For p≫ n, correlation screening [Fan and Lv (2008)] requires Ao to be a subset of the indices of the m largest values of |x′jy|/‖xj‖ with a certain m≤ n.”
Testing whether linear equations are causal: A free probability theory approach (2011, methodology): “As proposed in Fan and Lv (2008), in this very high dimensional case, Ai can be estimated by a two-scale scheme: a crude large scale screening procedure followed by a moderate scale variable selection. The screening techniques, such as sure independence screening (SIS) by Fan and Lv (2008), reduce n”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, background): “The vast majority of work has been pursued for point estimation such as consistency for prediction [7, 21], oracle inequalities and estimation of a highdimensional parameter [6, 11, 12, 24, 33, 34, 47, 51] or variable selection [17, 30, 49, 53].”
High-dimensional additive modeling (2008, methodology): “o [28] are now well understood [3, 14, 23, 24, 33] and this knowledge has led to several extensions or alternative approaches like adaptive lasso [34], relaxed lasso [22], sure independence screening [12] and graphical model based methods [6]. Moreover, with the fast growing amount of high-dimension”
Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models (2009, methodology): “…1 (s = 3,SNR ≈ 1.01) 3 (0) 3 (0) 3 (0) Example 1 (s = 6,SNR ≈ 1.99 ) 56 (0) 1000 (0) 56 (0) Example 1 (s = 12,SNR ≈ 4.07) 66 (7) 1000 (0) 62 (1) Example 1 (s = 24,SNR ≈ 8.20) 269 (134) 1000 (0) 109 (43) Example 2 (SNR ≈ 0.83) 3 (0) 3 (0) 360 (361) ranking method of Fan and Lv (2008).”
Preconditioning the Lasso for sign consistency (2015, background): “…(e.g. Lasso, SCAD, MCP), then the solution to the preconditioned penalized least squares problem selects the same sequence of models as preconditioned correlation screening (i.e. select X j if | cor ( , FX j ) ≥ λ ) (Fan and Lv, 2008). are consistent | Theorem 1 implies that all such methods sign .”
Conditional quantile screening in ultrahigh-dimensional heterogeneous data (2015, methodology): “To overcome the difficulties associated with ultrahigh dimensionality, Fan & Lv (2008) proposed a sure independent screening method to reduce the dimension, so that penalized variable selection procedures would be applicable.”
Fast Computing for Distance Covariance (2014, methodology): “The screening algorithm, which was initially advocated by Fan and Lv (2008), works as follows. For each covariateXk,1 ≤ k ≤ n, a ‘marginal utility’ function was computed. Such a marginal utility function can be the Pearson’s correlation, the distance correlation that was discussed in this paper, or ”
High Dimensional Classification Using Features Annealed Independence Rules. (2007, methodology): “siﬁcation, resulting in the nearest shrunken centroids method. Such a simple method corresponds to a 3 componentwise regression method or a ridge regression method with ridge parameters tending to ∞ (Fan and Lv, 2007). Hence, it is a speciﬁc and useful example of the penalized quasi-likelihood metho”
Oracle Inequalities for High Dimensional Vector Autoregressions (2012, methodology): “…examples are the SCAD of Fan and Li (2001), the adaptive LASSO of Zou (2006), the Bridge and Marginal Bridge estimators of Huang et al. (2008), the Dantzig selector of Cand`es and Tao (2007), the Sure Independence Screening of Fan and Lv (2008) and the square root LASSO of Belloni et al. (2011).”
Quantile Correlations and Quantile Autoregressive Modeling (2012, neutral): “…diagnosis of nonlinear quantile autoregression models (e.g., Chen, Koenker, and Xiao 2009) and quantile GARCH models (Xiao and Koenker 2009) can be considered; variable screening and selection (e.g., Fan and Lv 2008; Wang 2009) in quantile regressions is another important topic for future research.”
Estimation and model selection in generalized additive partial linear models for correlated data with diverging number of covariates (2014, methodology): “These bounds are assumed for the parametric models [8] or can be derived for independent observations [31] using Lemma 1 of [24].”
The Dantzig Selector in Cox's Proportional Hazards Model (2009, positive): “…data set and one way to reach an upper bound on p that achieves this is to extend and use a concept similar to that of sure screening proposed recently by Fan & Lv (2008) in the linear regression setting to reduce high dimensionality to a relatively smaller scale, possibly below the sample size.”
Using distance covariance for improved variable selection with application to learning genetic risk models (2014, methodology): “Reference [1] proposed SIS procedure based on the Pearson correlation for feature selection.”
A Model-Averaging Approach for High-Dimensional Regression (2014, methodology): “2011), the Dantzig selector (Candes and Tao 2007), marginal regression (Genovese, Jin, and Wasserman 2009), correlation learning (Fan and Lv 2008), and the partial faithfulness approach (Buhlmann, Kalisch, and Maathuis 2010) among many others.”