Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models

By Han Liu, Kathryn Roeder, Larry A. Wasserman. Neural Information Processing Systems, 2010

526 citations recorded by the source corpus.

A challenging problem in estimating high-dimensional graphical models is to choose the regularization parameter in a data-dependent way. The standard techniques include K-fold cross-validation (K-CV), Akaike information criterion (AIC), and Bayesian information criterion (BIC). Though these methods work well for low-dimensional problems, they are not suitable in high dimensional settings. In this paper, we present StARS: a new stability-based method for choosing the regularization parameter in high dimensional inference for undirected graphs. The method has a clear interpretation: we use the least amount of regularization that simultaneously makes a graph sparse and replicable under random sampling. This interpretation requires essentially no conditions. Under mild conditions, we show that StARS is partially sparsistent in terms of graph estimation: i.e. with high probability, all the true edges will be included in the selected model even when the graph size diverges with the sample size. Empirically, the performance of StARS is compared with the state-of-the-art model selection procedures, including K-CV, AIC, and BIC, on both synthetic data and a real microarray dataset. StARS outperforms all these competing procedures.

Subfields: Molecular Biology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3fc17a36e2bec28396fd17c1ec2288d817564565

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Estimation Stability With Cross-Validation (ESCV) (2013, background): “…of our work, recent follow-up papers on (Meinshausen and B¨uhlmann 2010) use model selection stability to select edges in graphical models (Liu et al. 2010; Haury et al. 2012) or modify stability model selection to improve its false discovery rate theoretical properties (Shah and…”
Sparse and Compositionally Robust Inference of Microbial Ecological Networks (2014, methodology): “Here we use a popular model selection scheme known as the Stability Approach to Regularization Selection (StARS) [48].”
A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression (2014, methodology): “In practice, it can be chosen using various methods, for example, the stability-based approach StARS (Meinshausen and Bühlmann, 2010; Liu et al., 2010; Shah and Samworth, 2013).”
Structure Learning in Graphical Modeling (2016, methodology): “A resampling technique that seeks to avoid false negatives was proposed by Liu et al. (2010).”
Controlling the false discovery rate via knockoffs (2014, methodology): “Finally, the stability selection approach [11, 13] controls false variable selection in the Lasso by refitting the Lasso model repeatedly for subsamples of the data, and then keeps only those variables that appear consistently in the Lasso model across most of the subsamples.”
mgm: Estimating Time-Varying Mixed Graphical Models in High-Dimensional Data (2015, methodology): “From a graphical models perspective this means that we estimate the neighborhood N ( s ) of each node s ∈ V and then combine all neighborhoods to obtain an estimate of the graph G (Meinshausen and B¨uhlmann 2006).”
Finding and Leveraging Structure in Learning Problems (2012, methodology): “Recently, there have been some promising methods for tuning parameter selection based on stability and bootstrapping [Meinshausen and Bühlmann, 2010, Liu et al., 2010] which can select sparsistent models from finite samples.”
High-Dimensional Statistics with a View Toward Applications in Biology (2014, methodology): “We refer the interested reader to the vignette of the huge package for more details on the use of this package. library(huge) set.seed(123) ## For ease of reproduction, we only use the 100 genes ## with largest empirical variance ## The analysis on the full data takes about 2 hours ## Apply nonparan”
Bayesian model choice and information criteria in sparse generalized linear models (2011, methodology): “As for regression, we can alternatively use stability selection (Meinshausen and Bühlmann, 2010), where we search for edges that are stable across sparse models fitted to subsamples of the data using graphical Lasso or neighborhood selection; see also Liu et al. (2010). This method has”
Network analysis methods for studying microbial communities: A mini review (2021, methodology): “SPIEC-EASI infers the appropriate amount of sparsity of a network by using the Stability Approach to Regularization Selection [46].”