© Institute of Mathematical Statistics, 2009 HIGH-DIMENSIONAL VARIABLE SELECTION BY LARRY WASSERMAN AND KATHRYN ROEDER 1

By Larry Wasserman, Kathryn Roeder

630 citations recorded by the source corpus.

This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high dimensional models? In particular, we look at the error rates and power of some multi-stage regression methods. In the first stage we fit a set of candidate models. In the second stage we select one model by cross-validation. In the third stage we use hypothesis testing to eliminate some variables. We refer to the first two stages as “screening” and the last stage as “cleaning.” We consider three screening methods: the lasso, marginal regression, and forward stepwise regression. Our method gives consistent variable selection under certain conditions.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Multi-population GWA mapping via multi-task regularized regression (2010, background): “For lasso regression, Wasserman and Roeder (2009) prove that the true association markers will be included in the predicted associated markers found by this validation procedure with high probability.”
A significance test for forward stepwise model selection (2014, background): “Most of these make use of subsampling (Meinshausen and Bühlmann, 2010) or data splitting (Wasserman and Roeder, 2009).”
Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models (2010, methodology): “In regression, cross-validation has been shown to overfit [22].”
Covariate‐assisted ranking and screening for large‐scale two‐sample inference (2019, methodology): “Tukey (1994) coined two terms to advocate some of his favourite information integration strategies: borrowing strength and pooling within.”
Why adaptively collected data have negative bias and how to correct for it (2017, background): “Wasserman and Roeder [23] and Meinshausen et al. [15] discussed data splitting in high-dimensional inference.”
Two-Stage Procedures for High-Dimensional Data (2011, methodology): “and Wasserman and Roeder (2009) considered variable selection for the regression model by using hypothesis testing to eliminate some variables such as”
High-dimensional simultaneous inference with the bootstrap (2016, background): “For general models and high-dimensional settings, sample splitting procedures (Wasserman and Roeder 2009; Meinshausen et al. 2009) and stability selection (Meinshausen and Bühlmann 2010; Shah and Samworth 2013) provide some statistical error control and significance.”
Valid confidence intervals for post-model-selection predictors (2014, result): “, Leeb and Potscher (2005), Potscher (2009), Potscher and Schneider (2010), and Schneider (2016). These references also point to numerous earlier results. Also, the work of Lockhart et al. (2014), Wasserman and Roeder (2009), and Wasserman (2014) should be mentioned here. For the LASSO, in particula”
Independent filtering increases detection power for high-throughput experiments (2010, background): “Conceptually similar screening approaches have also been proposed for variable selection in high-dimensional regression models (13, 14).”
Controlling false positive selections in high-dimensional regression and causal inference (2013, result): “Such a result is implicitly contained in the study of Wasserman and Roeder.22 Some discussion about the conditions used is given below.”
Causal Interaction in Factorial Experiments: Application to Conjoint Analysis (2018, neutral): “Another possible inferential approach is sample splitting where we collapse levels and select factors using training data and then estimate and compute confidence intervals for the AMEs and AMIEs using test data (Wasserman and Roeder 2009; Athey and Imbens 2016; Chernozhukov et al. 2018).”
A SIGNIFICANCE TEST FOR THE LASSO (2013, methodology): “Wasserman & Roeder (2009) propose a procedure for variable selection and p-value estimation based on sample splitting, and this was extended by Meinhasuen et al. (2009).”
Confidence intervals for high-dimensional inverse covariance estimation (2014, methodology): “Further methodology for inference for the regression coefficients in highdimensional regression includes methods based on sample splitting [22, 36], bootstrapping approach [9, 8], inference after variable selection [3] and other [20, 2].”
Two-Step Lasso Estimation of the Spatial Weights Matrix (2015, methodology): “Recent contributions include a Lasso signiﬁcance tests due to Lockhart et al. [51] and the sample splitting approaches proposed by Wasserman and Roeder [52] and Meinshausen et al. [53] which allow for controlling the false discovery rate.”
Handbook of Big Data (2016, background): “The basic operation is zooming and panning of a table that is presented in a graphical form, here called a blockplot .”