Philosophy and the practice of Bayesian statistics

By Andrew Gelman, Cosma Rohilla Shalizi. British Journal of Mathematical & Statistical Psychology, 2010

716 citations recorded by the source corpus.

A substantial school in the philosophy of science identifies Bayesian inference with inductive inference and even rationality as such, and seems to be strengthened by the rise and practical success of Bayesian statistics. We argue that the most successful forms of Bayesian statistics do not actually support that particular philosophy but rather accord much better with sophisticated forms of hypothetico-deductivism. We examine the actual role played by prior distributions in Bayesian models, and the crucial aspects of model checking and model revision, which fall outside the scope of Bayesian confirmation theory. We draw on the literature on the consistency of Bayesian updating and also on our experience of applied work in social science. Clarity about these matters should benefit not just philosophy of science, but also statistical practice. At best, the inductivist view has encouraged researchers to fit and compare models without checking them; at worst, theorists have actively discouraged practitioners from performing model checking because it does not fit into their framework.

Subfields: Cognitive Neuroscience, History and Philosophy of Science, Philosophy

View Paper: https://www.semanticscholar.org/paper/dc7002636b48f47a6ce6b214e1466916282c40be

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In History and Philosophy of Science

In Philosophy

Citations in the corpus

Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors (2020, background): “Bayesian inference is known to be poorly-calibrated under model misspecification (Gelman and Shalizi, 2013).”
Inference Compilation and Universal Probabilistic Programming (2016, background): “, 2015)—is the possibility of model misspecification (Shalizi et al., 2009; Gelman and Shalizi, 2013).”
Improving Transparency and Replication in Bayesian Statistics: The WAMBS-Checklist (2017, neutral): “…have an impact on parameter estimates and therefore also on substantive findings; for details on the different ways in which priors can adversely impact findings, see: Depaoli (2013); Gelman and Shalizi (2013); Johnson (2013); Seaman, Seaman, and Stamey (2012); and van de Schoot and Depaoli (2014).”
A guide to Bayesian model selection for ecologists (2015, background): “…arguments pertaining to model selection and multimodel inference we refer the interested reader to several excellent sources, including Gelman and Shalizi (2012) and J. M. Ver Hoef and P. L. Boveng (unpublished manuscript), who discuss when and why one should use model selection…”
Bayesian estimation supersedes the t test. (2013, background): “Confidence intervals (CIs) carry no distributional information, and therefore render power to be virtually unknowable because of its uncertainty.”
The Prior Can Often Only Be Understood in the Context of the Likelihood (2017, background): “Incoherence is an unavoidable aspect of much real-world data analysis; and, indeed, one might argue that as scientists we learn the most from the anomalies and reassessments associated with episodes of incoherence [8,9].”
For and Against Methodologies: Some Perspectives on Recent Causal and Statistical Inference Debates (2017, methodology): “…into fitted models, frequentist methods are used to test (calibrate) those models against data, and Bayesian methods are used to generate model-conditional inferences (e.g., see Box 1980; Dawid 1982; Rubin 1991; Little 2006; Gustafson and Greenland 2009; Greenland 2010b; Gelman and Shalizi 2013).”
Using Lasso for Predictor Selection and to Assuage Overfitting: A Method Long Overlooked in Behavioral Sciences (2015, background): “Although regularization is not necessarily conducted within a Bayesian framework, a conceptual link between regularization and Bayesian methods can be made (Gelman & Shalizi, 2013).”
Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models (2014, methodology): “Moreover, if we think of our model as a “working theory” about the data, these methods connect to ideas of falsification in the philosophy of science (Popper, 1959; Gelman and Shalizi, 2012).”
Beyond subjective and objective in statistics (2015, background): “Gelman and Shalizi (2013) incorporate this in an outline of what we refer to here as “falsiﬁcationist Bayesianism,” a philosophy that openly deviates from both objectivist and subjectivist Bayesianism, integrating Bayesian methodology with an interpretation of probability that can be seen as…”
Bayesian Estimation and Inference (2015, methodology): “The uncertainty (i.e., variance) in posterior distributions that are used for inferences is reduced when informed priors are consistent with the likelihood derived from data (see Figure 2b; Gelman & Shalizi, 2012), helping hypothesis tests.”
Moving beyond noninformative priors: why and how to choose weakly informative priors in Bayesian analyses (2019, background): “Bayesian linear regression and some simple hierarchical models have analytical solutions under conjugate priors, such that MCMC samplers are not necessary (Gelman et al. 2013).”
Default Bayes Factors for Model Selection in Regression (2012, methodology): “Arguments for the usefulness of subjective probability are made most elegantly in the psychological literature by Edwards, Lindman, and Savage (1963), and the interested reader is referred there.”
Bayesian Analysis of Factorial Designs (2017, background): “The fact that Bayes factors are dependent on prior settings has fueled the critique that they are too subjective (e.g., Gelman & Shalizi, 2013; Kruschke, 2013; Liu & Aitkin, 2008).”
Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It (2014, methodology): “, Gelman and Shalizi (2012)): even if all models are wrong, the information-theoretic MDL interpretation stands.”