A weakly informative default prior distribution for logistic and other regression models

By Andrew Gelman, Aleks Jakulin, M. G. Pittau, Yu-Sung Su. 2008

2,029 citations recorded by the source corpus.

We propose a new prior distribution for classical (nonhierarchical) logistic regression models, constructed by first scaling all nonbinary variables to have mean 0 and standard deviation 0.5, and then placing independent Student-t prior distributions on the coefficients. As a default choice, we recommend the Cauchy distribution with center 0 and scale 2.5, which in the simplest setting is a longer-tailed version of the distribution attained by assuming one-half additional success and one-half additional failure in a logistic regression. Cross-validation on a corpus of datasets shows the Cauchy class of prior distributions to outperform existing implementations of Gaussian and Laplace priors. We recommend this prior distribution as a default choice for routine applied use. It has the advantage of always giving answers, even when there is complete separation in logistic regression (a common problem, even when the sample size is large and the number of predictors is small), and also automatically applying more shrinkage to higher-order interactions. This can be useful in routine data analysis as well as in automated procedures such as chained equations for missing-data imputation. We implement a procedure to fit generalized linear models in R with the Student-t prior distribution by incorporating an approximate EM algorithm into the usual iteratively weighted least squares. We illustrate with several applications, including a series of logistic regressions predicting voting preferences, a small bioassay experiment, and an imputation model for a public health data set.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Coresets for Scalable Bayesian Logistic Regression (2016, methodology): “The analysis in this work allows any prior π0(θ); common choices are the Gaussian, Cauchy [12], and spike-and-slab [13].”
Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach (2014, methodology): “We use the preprocessing and Cauchy prior recommended by Gelman et al. (2008).”
Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors (2014, background): “Therefore, our approach is closer in spirit of weakly informative priors defended by Gelman (2006); Gelman et al. (2008), although, should the user require it, there is no reason that these priors can’t be truly informative.”
Bayesian Learning with Wasserstein Barycenters (2018, methodology): “Setting a flat prior [Gelman et al., 2008] over the hyperparameters, we trained a Gaussian process (GP) [Rasmussen and Williams, 2006] with constant mean function and cosine covariance kernel.”
Flexible Imputation of Missing Data (2012, methodology): “Alternative methods for right-censored data have also been proposed (Wei and Tanner, 1991; Geskus, 2001; Lam et al., 2005; Liu et al., 2011). Lyles et al. (2001), Lynn (2001) and Hopke et al. (2001) concentrated on left-censored data.”
Statistical Analysis of List Experiments (2012, background): “In such situations, weakly informative priors can be added to the models for ceiling/floor effects by following the recommendations of Gelman et al. (2008) and calculating the uncertainty estimates based on this pseudoposterior distribution.”
Visualization in Bayesian workflow (2017, background): “The prior distribution over the data enables us to extend the concept of a weakly informative prior (Gelman et al., 2008) to be more aware of the role of the likelihood.”
Deep Evidential Regression (2019, background): “A large topic of research in Bayesian inference focuses on placing prior distributions over hierarchical models to estimate uncertainty (Gelman et al., 2006; 2008).”
Sample size for binary logistic prediction models: Beyond events per variable criteria (2018, methodology): “Some more recent refinements to the Firth’s correction have shown promising results in circumventing the issues with calibration in the large.(21,48,58,59) With larger sample sizes, the benefits (in terms of predictive performance) of the regression shrinkage strategies gradually declined, but predi”
Territoriality, Social Bonds, and the Evolution of Communal Signaling in Birds (2016, positive): “For all BTMMs and BPMMs, we used a Gelman prior for random eﬀects (in MCMCglmm (Hadﬁeld, 2010) using the command “gelman.prior” (Gelman et al., 2008) with V = 10 − 6 , nu = − 1).”