Variable selection with error control: another look at stability selection

By Rajen Dinesh Shah, Richard J. Samworth. 2011

406 citations recorded by the source corpus.

Summary. Stability selection was recently introduced by Meinshausen and Bühlmann as a very general technique designed to improve the performance of a variable selection algorithm. It is based on aggregating the results of applying a selection procedure to subsamples of the data. We introduce a variant, called complementary pairs stability selection, and derive bounds both on the expected number of variables included by complementary pairs stability selection that have low selection probability under the original procedure, and on the expected number of high selection probability variables that are excluded. These results require no (e.g. exchangeability) assumptions on the underlying model or on the quality of the original selection procedure. Under reasonable shape restrictions, the bounds can be further tightened, yielding improved error control, and therefore increasing the applicability of the methodology.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Forward stability and model path selection (2021, neutral): “…(SS) is one popular resampling-based method for estimating S that was introduced by Meinshausen and Buhlmann (2010) and has since been reﬁned by Shah and Samworth (2013) For each m ∈ { 1 , . . . , M } we then calculate the frequency of selection for each covariate across all resamples so that…”
On asymptotically optimal confidence regions and tests for high-dimensional models (2013, background): “Stability selection [31] and its modification [41] provides another route to estimate error measures for false positive selections in general high-dimensional settings.”
High-Dimensional Statistics with a View Toward Applications in Biology (2014, background): “The work in Shah and Samworth (2013) extends the result in (10) without requiring the restrictive but not necessary exchangeability assumption.”
Sparse Estimation with Strongly Correlated Variables using Ordered Weighted L1 Regularization (2014, unknown): “Stability selection procedures, can also aid in the selection of correlated columns, as shown by Meinshausen and Bülhmann [15], and Shah and Samworth [19].”
A Split-and-Conquer Approach for Analysis of Extraordinarily Large Data (2014, background): “According to Singh et al. (2005), Xie et al. (2011) and Liu (2012), equivalent combined statistics or asymptotic efficiency are achievable for many other models.”
Structure Learning in Graphical Modeling (2016, background): “Another useful approach is stability selection (Meinshausen & B¨uhlmann, 2010; Shah & Samworth, 2013).”
Estimation Stability With Cross-Validation (ESCV) (2013, background): “Shah and Samworth (2013) employs the complementary half-sample data perturbation scheme.”
High dimensional ordinary least squares projection for screening variables (2015, methodology): “For example, the Lasso can give non-consistent models if the irrepresentable condition on the design matrix is violated (Zhao and Yu, 2006; Zou, 2006), although computationally more extensive methods such as those combining subsampling and structure selection (Meinshausen and Bühlmann, 2010; Shah an”
A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models (2014, unknown): “For instance, in the linear model, Wang et al. (2013b) showed that ‖β̂ − β∗‖∞ = OP( √ log s∗ n ), with the nonconvex penalty.”
Variable selection using pseudo-variables (2018, methodology): “…Zhang et al., 2010; Fan and Tang, 2013; Hui et al., 2015), resampling methods (Hall et al., 2009; Meinshausen and Bühlmann, 2010; Feng and Yu, 2013; Sun et al., 2013; Shah and Samworth, 2013; Sabourin et al., 2015), and variable addition methods (Wu et al., 2007; Barber and Candès, 2015, 2016).”
A general framework for functional regression modelling (2017, methodology): “Variable selection results from both early stopping—not all base learners are selected in at least one of the iterations—and additional stability selection (Meinshausen and Bühlmann, 2010; Shah and Samworth, 2013), which is based on the stability of base learner selection under sub-sampling.”
Goodness‐of‐fit tests for high dimensional linear models (2015, unknown): “nn, 2010] and its variant Complementary Pairs Stability Supported in part by the Forschungsintitut fur Mathematik (FIM) at ETH Zuric h. 1 arXiv:1511.03334v1 [stat.ME] 10 Nov 2015 Selection (CPSS) [Shah and Samworth, 2013] can also be used to select important variables whilst preserving error control”
A Unified Theory of Confidence Regions and Testing for High-Dimensional Estimating Equations (2015, unknown): “P-values and confidence intervals based on sample splitting and subsampling are suggested by Meinshausen et al. (2009); Meinshausen and Bühlmann (2010); Shah and Samworth (2013); Wasserman and Roeder (2009).”
Group bound: confidence intervals for groups of variables in sparse high dimensional regression without assumptions on the design (2013, background): “ecently received substantial attention. While the overall stability of estimated sparse solutions was sometimes analysed and exploited for better structure discovery [Meinshausen and Buhlmann, 2010, Shah and Samworth, 2013, Lim and Yu, 2013], formal signicance tests were provided in Wasserman and Ro”
High-dimensional simultaneous inference with the bootstrap (2016, background): “2009) and stability selection (Meinshausen and Bühlmann 2010; Shah and Samworth 2013) provide some statistical error control and significance.”
Splitting strategies for post-selection inference (2021, unknown): “R package of Tibshirani et al. (2019), which estimates σ2 using the residual sum of squares from a lasso fit with the penalty parameter tuned by cross-validation.”
A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression (2014, methodology): “In practice, it can be chosen using various methods, for example, the stability-based approach StARS (Meinshausen and Bühlmann, 2010; Liu et al., 2010; Shah and Samworth, 2013).”