A variational Bayes approach to variable selection
By John Thomas Ormerod, Chong You, Samuel Müller. 2017
65 citations recorded by the source corpus.
We develop methodology and theory for a mean field variational Bayes approximation to a linear model with a spike and slab prior on the regression coefficients. In particular we show how our method forces a subset of regression coefficients to be numerically indistinguishable from zero; under mild regularity conditions estimators based on our method consistently estimates the model parameters with easily obtainable and appropriately sized standard error estimates; and, most strikingly, selects the true model at a exponential rate in the sample size. We also develop a practical method for simultaneously choosing reasonable initial parameter values and tuning the main tuning parameter of our algorithms which is both computationally efficient and empirically performs as well or better than some popular variable selection approaches. Our method is also faster and highly accurate when compared to MCMC.
Subfields: Artificial Intelligence, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- David M. Blei
- Alp Kucukelbir
- Jon D. McAuliffe
- Stephan Mandt
- Rajesh Ranganath
- B. Naughton
- Y. Zhang
- Howard D. Bondell
In Statistics and Probability
- Alp Kucukelbir
- Jon D. McAuliffe
- B. Naughton
- David M. Blei
- Y. Zhang
- Howard D. Bondell
- Brian J. Reich
- Rajesh Ranganath
Citations in the corpus
- Variational Inference: A Review for Statisticians (2016, background): “Ormerod, You, and Muller (2014) built on their earlier work with a spike-and-slab prior on the coefficients and found similar consistency results.”
- Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior (2016, methodology): “ead can proceed instead by applying approximation methods, such as stochastic search variable selection (George & McCulloch 1993), shotgun stochastic search (Hans et al. 2007), variational Bayes (Ormerod et al. 2017), and EM (Rockova & George 2014) all of which have improved the computationa”