A review of Bayesian variable selection methods: what, how and which

By R. O’Hara, Mikko J. Sillanpää. 2009

812 citations recorded by the source corpus.

The selection of variables in regression problems has occupied the minds of many statisticians. Several Bayesian variable selection methods have been developed, and we concentrate on the following methods: Kuo & Mallick, Gibbs Variable Selection (GVS), Stochastic Search Variable Selection (SSVS), adaptive shrinkage with Jefireys' prior or a Laplacian prior, and reversible jump MCMC. We review these methods, in the context of their difierent properties. We then implement the methods in BUGS, using both real and simulated data as examples, and investigate how the difierent methods perform in practice. Our results suggest that SSVS, reversible jump MCMC and adaptive shrinkage methods can all work well, but the choice of which method is better will depend on the priors that are used, and also on how they are implemented.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b53f4a5d57762d42b5a34889c64d4bdad79b345f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Individual heterogeneity and capture–recapture models: what, why and how? (2018, methodology): “We refer to O’Hara and Sillanpää (2009) for Bayesian methods (see also Royle 2008, Chambert et al. 2014).”
Bayesian variable selection regression for genome-wide association studies and other large-scale problems (2011, background): “A P] 2 7 O ct 2 01 1 The Annals of Applied Statistics 2011, Vol. 5, No. 3, 1780–1815 DOI: 10.1214/11-AOAS455 c© Institute of Mathematical Statistics, 2011 BAYESIAN VARIABLE SELECTION REGRESSION FOR”
A Default Bayesian Hypothesis Test for ANOVA Designs (2012, background): “…are rarely conducted in ANOVA designs; instead, most theoretical development has concerned the more general problem of selecting variables in regression models (e.g., Mitchell and Beauchamp 1988; George and McCulloch 1997; Kuo and Mallick 1998; Casella and Moreno 2006; O’Hara and Sillanpää 2009).”
Comparison of Bayesian predictive methods for model selection (2015, methodology): “(see, e.g., George and McCulloch 1993, 1997; Raftery et al. 1997; Brown et al. 2002; Lee et al. 2003; O’Hara and Sillanpää 2009; Narisetty and He 2014) and extended and applied to regression for over one million variables (Peltola et al.”
Bayesian Hierarchical Models With Conjugate Full-Conditional Distributions for Dependent Data From the Natural Exponential Family (2017, methodology): “Spike and slab, horseshoe priors, and SSVS (among other similar techniques) are extensions of the LGP, which one might adapt to the LCM to select covariates and basis functions (OHara and Sillanpaa, 2009); however, we do not consider these extensions of the LCM in this article.”
Review and evaluation of penalised regression methods for risk prediction in low‐dimensional data with few events (2015, background): “O’Hara and Sillanpaa [38] provide a nice review of the Bayesian approach to model selection.”
A Unified Framework for Fitting Bayesian Semiparametric Models to Arbitrarily Censored Survival Data, Including Spatially Referenced Data (2017, neutral): “The resulting MCMC algorithm does not require any tuning, but mixing can be poor if the prior on β is too diffuse (O’Hara and Sillanpää, 2009).”
Towards novel approaches to modelling biotic interactions in multispecies assemblages at large spatial extents (2012, methodology): “…(Table 2), including approximations for spatial processes (e.g. Banerjee et al., 2008), model selection procedures (Burnham & Anderson, 2002; O’Hara & Sillanpää, 2009), or shrinkage methods, which simultaneously estimate parameters whilst shrinking the unimportant ones towards zero…”
Polynomial nonlinear spatio‐temporal integro‐difference equation models (2011, background): “The Bayesian variable selection literature is extensive, with many variations on these basic methods having been proposed (e.g. see O’Hara and Sillanpää (2009) for a comprehensive review).”
Polygenic Modeling with Bayesian Sparse Linear Mixed Models (2012, background): “It is unclear whether this will necessarily result in a loss of computational efficiency: sampling ~ b reduces computational expense per update at the cost of increasing the number of updates necessary (sampling c by integrating over ~ b analytically ensures faster mixing and convergence [74,75]).”