Journal of the American Statistical Association Bayesian Inference for Logistic Models Using Pólya–gamma Latent Variables Bayesian Inference for Logistic Models Using P ´ Olya–gamma Latent Variables

By Nicholas G. Polson, James G. Scott, Jesse Windle

1,175 citations recorded by the source corpus.

We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Pólya–Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negative binomial regression, nonlinear mixed-effect models, and spatial models for count data. In each case, our data-augmentation strategy leads to simple, effective methods for posterior inference that (1) circumvent the need for analytic approximations, numerical integration, or Metropolis–Hastings; and (2) outperform other known data-augmentation strategies, both in ease of use and in computational efficiency. All methods, including an efficient sampler for the Pólya–Gamma distribution, are implemented in the R package BayesLogit. Supplementary materials for this article are available online.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, methodology): “One appealing approach to address this obstacle is the Pólya-Gamma augmentation (Polson et al., 2013).”
Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes (2020, methodology): “1 Pólya-Gamma Augmentation The Pólya-gamma augmentation scheme was originally introduced to address Bayesian inference in logistic models [21] and has since been applied to multinomial GPs via a stick-breaking construction [15] and to GP-based classification with the logistic softmax likelihood [8].”
Dependent Multinomial Models Made Easy: Stick-Breaking with the Polya-gamma Augmentation (2015, methodology): “This augmentation scheme has been used to develop Gibbs sampling and variational inference algorithms for Bernoulli, binomial (Polson et al., 2013), and negative binomial regression models (Zhou et al., 2012) with logit link functions.”
Learning Deep Sigmoid Belief Networks with Data Augmentation (2015, methodology): “From recent work for the Pólya-Gamma data augmentation strategy (Polson et al., 2013), that is, if γ ∼ PG(b, 0), b > 0, then (e ψ ) a (1 + e ψ ) b = 2 −b e κψ ∞ 0 e −γψ 2 /2 p(γ)dγ , (10) where κ = a − b/2.”
On Perturbed Proximal Gradient Algorithms (2014, methodology): “The distribution πθ is sampled using the MCMC sampler proposed in Polson et al. (2013) based on data-augmentation.”
Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation (2019, methodology): “In the following section we derive a three steps augmentation scheme, where we (i) decouple the GP latent variables f i in the denominator by the introduction of a set of auxiliary λ-variables, (ii) further simplify the model likelihood by introducing Poisson random variables, and finally (iii) use ”
Semi-Implicit Variational Inference (2018, background): “The contribution of observation i to the likelihood can be expressed as where the expectation is taken respect to a P ´ olya-Gamma (PG) distribution (Polson et al., 2013) as ω i ∼ PG (1 , 0) , and hence we have an augmented likelihood as”
On stochastic proximal gradient algorithms (2014, background): “…this discussion evidences that when the bias is not zero, the computational cost to reach the accuracy δ is again in favor of the basic stochastic proximal gradient algorithm (compare the rate δ −2 for the basic algorithm to δ −2 (− log δ) β for the accelerated one with constant step size).”
Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation (2018, methodology): “Gaussian processes (GPs) Rasmussen and Williams (2005) provide a popular Bayesian non-linear non-parametric method for regression and classification. Because of their ability of accurately adapting to data and thus achieving high prediction accuracy while providing well calibrated uncertainty estima”
Variational Inference for Sparse and Undirected Models (2016, methodology): “hoe (Carvalho et al.,2010) both incorporate these properties, but unfortunately hinder MAP inference with non-convexities and typically require the use of either HMC or augmentation-based approaches (Polson et al., 2013). Applying these favorable, Bayesian approaches to sparsity has been particularl”
Efficient posterior sampling for high-dimensional imbalanced logistic regression. (2019, methodology): “A popular algorithm for sampling from π ( ξ ) is P´olya-Gamma data augmentation [25].”
Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling (2018, methodology): “…models that strictly generalizes the sequential stick breaking of the original rSLDS, while still permitting Pólya-gamma data augmentation (Polson et al., 2013) for efﬁcient posterior inference; second, we develop a hierarchical prior that links dynamics parameters across levels of the…”
Flexible statistical inference for mechanistic models of neural dynamics (2017, methodology): “We evaluated the posterior distribution over the parameters of a GLM using a Pólya-Gamma sampler (PG-MCMC, [42, 43]) and compared it to the posterior distributions estimated by SNPE (Appendix D.”
Boosting Variational Inference (2016, methodology): “We treat results from a Polya-Gamma sampler (a specialized MCMC algorithm) using R package BayesLogit [Polson et al., 2013] as the ground truth.”