Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models

By Jianqing Fan, Yang Feng, Rui Song. 2009

602 citations recorded by the source corpus.

A variable screening procedure via correlation learning was proposed by Fan and Lv (2008) to reduce dimensionality in sparse ultra-high-dimensional models. Even when the true model is linear, the marginal regression can be highly nonlinear. To address this issue, we further extend the correlation learning to marginal nonparametric learning. Our nonparametric independence screening (NIS) is a specific type of sure independence screening. We propose several closely related variable screening procedures. We show that with general nonparametric models, under some mild technical conditions, the proposed independence screening methods have a sure screening property. The extent to which the dimensionality can be reduced by independence screening is also explicitly quantified. As a methodological extension, we also propose a data-driven thresholding and an iterative nonparametric independence screening (INIS) method to enhance the finite- sample performance for fitting sparse additive models. The simulation results and a real data analysis demonstrate that the proposed procedure works well with moderate sample size and large dimension and performs better than competing methods.

Subfields: Artificial Intelligence, Molecular Biology, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Generalized Sparse Additive Models (2019, background): “There are also a number of other proposals that do not quite fall in the GSAM framework [Chouldechova and Hastie, 2015, Fan et al., 2012, Yin et al., 2012].”
A flexible semiparametric forecasting model for time series (2015, methodology): “…by the recent work on the sure independence screening for variable selection in high-dimensional feature space, where the marginal regression is used to choose relevant covariates, see, for example, Fan and Lv (2008) and Fan et al (2011) where the observations are assumed to be independent.”
Conditional quantile screening in ultrahigh-dimensional heterogeneous data (2015, methodology): “…have been studied extensively in various ultrahigh-dimensional contexts, such as generalized linear models (Fan & Song, 2010), additive models (Fan et al., 2011), Cox proportional hazards regression with survival data (Zhao & Li, 2012), and additive hazard models (Gorst-Rasmussen & Scheike,…”
Minimax Optimal Rates of Estimation in High Dimensional Additive Models: Universal Phase Transition (2015, background): “…e.g., Lin and Zhang (2006), Yuan (2007), Koltchinskii and Yuan (2008), Ravikumar et al. (2009), Meier, van de Geer and Bühlmann (2009), Huang, Horowitz and Wei (2010), Koltchinskii and Yuan (2010), Fan, Feng and Song (2011), Raskutti, Wainwright and Yu (2012), and Cui et al. (2013) among others.”
Independent screening for single‐index hazard rate models with ultrahigh dimensional features (2011, methodology): “Following Fan and Lv (2008) and Fan et al. (2009), we would typically choose γn such that |M̂nd | is of order n= log.n/. Devising a data-adaptive way of choosing γn is an open problem for independent screening methods in general and is beyond the scope of this paper. Suggestions were recently given ”
The fused Kolmogorov filter: A nonparametric model-free screening method (2014, methodology): “Since the sure independence screening paper by Fan and Lv (2008), variable screening has received a lot of attention in the literature and many variable screening techniques, both parametric and nonparametric, have been proposed and studied in recent years [Chang, Tang and Wu (2013), Fan and Fan (20”
Estimation and model selection in generalized additive partial linear models for correlated data with diverging number of covariates (2014, methodology): “With a total of 12,626 genes as covariates and only 54 observations, we first apply the nonparametric independence screening method (NIS) described in [6] to reduce the dimensionality to a moderate size.”
Semiparametric regression models with additive nonparametric components and high dimensional parametric components (2012, methodology): “For the parametric components, we use the SCAD penalty Fan and Li (2001).”
Journal of the American Statistical Association Linear or Nonlinear? Automatic Structure Discovery for Partially Linear Models Linear or Nonlinear? Automatic Structure Discovery for Partially Linear Models (positive): “For ultrahigh-dimensional data, we suggest to combine the LAND procedures with dimension reduction techniques such as Sure Independence Screening (Fan and Jinchi 2008; Fan, Feng, and Song 2011).”
Semiparametric Ultra-High Dimensional Model Averaging of Nonlinear Dynamic Time Series (2018, positive): “When the observations are independent and the response variable has zero mean, the article of Fan, Feng, and Song (2011) ranks the importance of the covariates by calculating the L2-norm of m̂ j(·), and chooses those covariates whose corresponding norms are larger than a predetermined threshold…”
Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data (2013, methodology): “In the present setup, we adopt the nonparametric independence screening (NIS) idea in [3].”
A Projection-based Conditional Dependence Measure with Applications to High-dimensional Undirected Graphical Models. (2015, methodology): “) We follow the implementation in Fan et al. (2009) for the three penalized likelihood estimators.”
Feature Screening via Distance Correlation Learning (2012, methodology): “Fan, Feng and Song ( 2011 ) considered nonparametric independence screening in sparse ultrahigh dimensional additive models.”
High dimensional ordinary least squares projection for screening variables (2015, background): “, 2009; Fan and Song, 2010), additive models (Fan et al., 2011) and hazard regression (Zhao and Li, 2012; Gorst-Rasmussen and Scheike, 2013) and to accommodate conditional correlation (Barut et al.”