Validation of Software for Bayesian Models Using Posterior Quantiles

By Samantha R. Cook, Andrew Gelman, Donald B. Rubin. 2006

308 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3d727ef56f07ab2406d0aeaaa18e51b932485fc0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Differentially Private Bayesian Inference for Exponential Families (2018, methodology): “For calibration we adapt a method of Cook et al. [30]: the idea is to draw iid samples ( θ i , x i ) from the joint model p ( θ ) p ( x | θ ) , and conduct posterior inference in each trial.”
A novel Bayesian method for inferring and interpreting the dynamics of adaptive landscapes from phylogenetic comparative data (2014, methodology): “If the posterior distribution is estimated correctly, then the distribution of these quantiles across simulations should follow a uniform distribution (Cook et al. 2006).”
Bayesian Workflow (2020, methodology): “1 is simulation-based calibration (SBC; Cook et al., 2006, Talts et al., 2020). In this scheme, the model parameters are drawn from the prior; then data are simulated conditional on these parameter values; then the model is fit to data; and finally the obtained posterior is compared to the simulated”
POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES (2011, methodology): “To check the correctness of the algorithm design and implementation we used the joint distribution tests from Geweke (2004) and related tests from Cook, Gelman, and Rubin (2006).”
Extracting distribution parameters from multiple uncertain observations with selection biases (2018, background): “If posteriors are self-consistent, we expect the truth to fall within the X% Bayesian credible interval X% of the time, i.e., the p-p plot should be diagonal (e.g., Cook et al. 2006; Sidery et al. 2014; Veitch et al. 2015).”
Likelihood-free MCMC with Approximate Likelihood Ratios (2019, methodology): “Earlier work [27] proposes a method to verify the Bayesian computation of software.”
MTML-msBayes: Approximate Bayesian comparative phylogeographic inference from multiple taxa and multiple loci with rate heterogeneity (2011, methodology): “For simulation-based testing, we generally compare estimates from PODS with the known hyper-parameter values that simulated the PODS [10,50] and calculate RMSE and RMSPE (root mean square error and root mean square posterior error) using these known values and each posterior mode estimate and the…”
Bayesian inference for compact binary coalescences with bilby: validation and application to the first LIGO–Virgo gravitational-wave transient catalogue (2020, background): “…algorithms is to check that the correct proportion of true parameter values are found within a given probability interval for simulated injections (Cook et al. 2006; Talts et al. 2018)—i.e. that 10% of events are found withing the 0.1 probability credible interval, 50% are found within the 0.5…”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, methodology): “ is much greater compared to the high capacity classiﬁers. In some regions the sample variance of the classiﬁer is 0, indicating a perfect approximation. 5.2 Simulation based calibration Earlier work [27] proposes a method to verify the Bayesian computation of software. The diagnostic is based on th”
Differentially Private Bayesian Linear Regression (2019, methodology): “We adapt a method of Cook et al. [2006] to measure calibration.”