Using Stacking to Average Bayesian Predictive Distributions (with Discussion)

By Yuling Yao, Aki Vehtari, Daniel P. Simpson, Andrew Gelman. Bayesian Analysis, 2017

424 citations recorded by the source corpus.

The widely recommended procedure of Bayesian model averaging is flawed in the M-open setting in which the true data-generating process is not one of the candidate models being fit. We take the idea of stacking from the point estimation literature and generalize to the combination of predictive distributions, extending the utility function to any proper scoring rule, using Pareto smoothed importance sampling to efficiently compute the required leave-one-out posterior distributions and regularization to get more stability. We compare stacking of predictive distributions to several alternatives: stacking of means, Bayesian model averaging (BMA), pseudo-BMA using AIC-type weighting, and a variant of pseudo-BMA that is stabilized using the Bayesian bootstrap. Based on simulations and real-data applications, we recommend stacking of predictive distributions, with BB-pseudo-BMA as an approximate alternative when computation cost is an issue.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/413199283cf82f4f8573491ac580df086596b945

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Incubation Period and Other Epidemiological Characteristics of 2019 Novel Coronavirus Infections with Right Truncation: A Statistical Analysis of Publicly Available Case Data (2020, methodology): “Weights can be interpreted as the probability of each model given the data and were calculated using stacking method [21].”
Forecast combinations: An over 50-year review (2022, background): “Palm and Zellner (1992) concisely summarized the advantages of adopting”
Learning to Average Predictively over Good and Bad: Comment on: Using Stacking to Average Bayesian Predictive Distributions (2018, background): “Yao, Vehtari, Simpson, and Gelman(2018) focus in their paper on the speciﬁc limitation of BMA when the true data generating process is not in the set.”
Pareto Smoothed Importance Sampling (2015, methodology): “PSIS has been used to improve and diagnose variational approximations to posteriors (Yao et al., 2018b; Magnusson et al., 2019, 2020; Dhaka et al., 2020, 2021; Zhang et al., 2021).”
Bayesian Workflow (2020, methodology): “Instead, we can maintain this information and use stacking to combine inferences using a weighting that is set up to minimize cross validation error (Yao et al., 2018b).”
Model averaging in ecology: a review of Bayesian, information-theoretic, and tactical approaches for predictive inference (2018, background): “Apart from the natural Bayesian average (see also Yao et al. 2017), there are a number of other approaches that are connected to or inspired by Bayesian thinking.”
Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors (2020, methodology): “…marginal-likelihood-based “full-Bayes” approach produces over-conﬁdent prediction when none of the models is true (Clarke, 2003; Wong and Clarke, 2004; Clyde and Iversen, 2013; Yao et al., 2018a; Yang and Zhu, 2018; Oelrich et al., 2020), and therefore is not Bayes optimal (Le and Clarke, 2017).”