Bayesian auxiliary variable models for binary and multinomial regression

By Christopher C. Holmes, Leonhard Held. 2006

516 citations recorded by the source corpus.

. In this paper we discuss auxiliary variable approaches to Bayesian binary and multinomial regression. These approaches are ideally suited to automated Markov chain Monte Carlo simulation. In the (cid:12)rst part we describe a simple technique using joint updating that improves the performance of the conventional probit regression algorithm. In the second part we discuss auxiliary variable methods for inference in Bayesian logistic regression, including covariate set uncertainty. Finally, we show how the logistic method is easily extended to multinomial regression models. All of the algorithms are fully automatic with no user set parameters and no necessary Metropolis-Hastings accept/reject steps.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/25e1dd4fe46d10389a0b16c87c992d3fdb4f370d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Time varying VARs with inequality restrictions (2011, methodology): “To evaluate this issue, we present the commonly-used e¤ective sample size (see, among many others, Holmes and Held, 2006 ), de…ned as:”
Sampling from a strongly log-concave distribution with the Unadjusted Langevin Algorithm (2016, methodology): “Several algorithms have been proposed, trying to mimick the data-augmentation (DA) approach of [1] for probit regression; see [15], [9] and [10].”
K-state switching models with time-varying transition distributions—Does loan growth signal stronger effects of variables on inflation? (2015, methodology): “Hence, we can rely on the Gibbs sampler while the alternative sampler of Holmes and Held (2006) involves rejection sampling in the random utility representation of the logit regression model.”
JAGS Version 3.3.0 user manual (2012, methodology): “This idea was introduced by Albert and Chib [1] for probit regression with a binary outcome, and was later refined and extended to logistic regression with binary outcomes by Holmes and Held [7].”
Stochastic model specification search for Gaussian and partial non-Gaussian state space models (2010, methodology): “This prior leads to a uniform distribution over model size and outperforms the uniform prior over all models in variable selection for large regression models, see Ley and Steel (2007). In our applications, where model size is small, posterior inference under both priors is virtually the same.”
Variational learning for latent Gaussian model of discrete data (2012, methodology): “The most popular approach is to use Markov chain Monte Carlo (MCMC) methods [Albert and Chib, 1993; Frühwirth-Schnatter and Frühwirth, 2010; Holmes and Held, 2006; Mohamed et al., 2008; Scott, 2011].”
A Bayesian Semiparametric Latent Variable Model for Mixed Responses (2007, unknown): “Recently, UVA-based Gibbs sampling for logit models has been suggested by O’Brien and Dunson (2004) and Holmes and Held (2005) .”
Provable Bayesian Inference via Particle Mirror Descent (2015, background): “Obviously, Gibbs sampling [20], which needs to scan the whole dataset, is not suitable for large-scale problem.”
Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm (2015, methodology): “to the analytically inconvenient form of the model’s likelihood function. Several algorithms have been proposed, trying to mimick the data-augmentation (DA) approach of [1] for probit regression; see [19], [13] and [14]. Recently, a very promising DA algorithm has been proposed in [28], using the Po”
High-dimensional Bayesian inference via the unadjusted Langevin algorithm (2016, methodology): “ression model has long been recognized as a numerically involved problem. Several algorithms have been proposed, trying to mimick the data-augmentation (DA) approach of [1] for probit regression; see [23], [18] and [19]. Recently, a very promising DA algorithm has been proposed in [36], using the Po”
Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (2009, unknown): “For some (observational) models, auxiliary variables can be introduced to simplify the construction of Gaussian approximations (Shephard, 1994; Albert and Chib, 1993; Holmes and Held, 2006; Frühwirth-Schnatter and Wagner, 2006; Frühwirth-Schnatter and Frühwirth, 2007; Rue and Held, 2005).”
Riemannian Manifold Hamiltonian Monte Carlo (2009, methodology): “The auxiliary variable Gibbs sampler of Holmes and Held (2005) was implemented with a joint update of{z, β}, wherez ∈ RN is the auxiliary variable designed to improve mixing of the covariate samples.”