Bayesian Versus Orthodox Statistics: Which Side Are You On?

By Zoltan Dienes. Perspectives on Psychological Science, 2011

959 citations recorded by the source corpus.

Researchers are often confused about what can be inferred from significance tests. One problem occurs when people apply Bayesian intuitions to significance testing—two approaches that must be firmly separated. This article presents some common situations in which the approaches come to different conclusions; you can see where your intuitions initially lie. The situations include multiple testing, deciding when to stop running participants, and when a theory was thought of relative to finding out results. The interpretation of nonsignificant results has also been persistently problematic in a way that Bayesian inference can clarify. The Bayesian and orthodox approaches are placed in the context of different notions of rationality, and I accuse myself and others as having been irrational in the way we have been using statistics on a key notion of rationality. The reader is shown how to apply Bayesian inference in practice, using free online software, to allow more coherent inferences from data.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c961147410f1117b4a4b6125a28b6ab3662471ac

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Multisensory Integration in Complete Unawareness (2014, methodology): “…assess whether the lack of priming stemmed from insufficient sensitivity of our experimental design or from a genuine lack of effect, we computed Bayes factors (referred to as B hereafter; see Jeffreys, 1961; Dienes, 2011; 0.33 < B < 3 suggests insensitivity, and B < 0.33 implies no effect).”
Using Bayes to get the most out of non-significant results (2014, background): “…with orthodox statistics, unless pre-specified as possible by one’s stopping rule (Armitage et al., 1969); by contrast, with Bayes, one can always collect more participants until the data are sensitive enough, that is, B < 1/3 or B > 3; see e.g., Berger and Wolpert (1988), Dienes (2008, 2011).”
Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison (2011, neutral): “For an overview of corrections for multiple comparisons in NHST, see Maxwell and Delaney (2004, chapter 5), and for a discussion of contrasting intuitions in multiple testing, see Dienes (2011).”
How Bayes factors change scientific practice (2016, background): “The paradoxes mean that inferential outcome depends on more than the actual data obtained, and may depend on things which are in practice unknowable (the intentions and thoughts of experimenters; see Dienes, 2011 for explanation).”
Bayes factor design analysis: Planning for compelling evidence (2016, methodology): “It is important to note that, in contrast to NHST, the inference drawn from the actual data set is entirely independent from the planning stage (Berger & Wolpert, 1988; Wagenmakers et al., 2014; Dienes, 2011).”
Improving Transparency and Replication in Bayesian Statistics: The WAMBS-Checklist (2017, neutral): “Second, many scholars prefer Bayesian statistics because they believe population parameters should be viewed as random (see, e.g., Dienes, 2011; van de Schoot et al., 2011).”
Bayesian alternatives for common null-hypothesis significance tests in psychiatry: a non-technical guide using JASP (2018, background): “Unlike classical inferential frameworks, Bayesian inference can incorporate such prior knowledge [17].”
Bayesian estimation supersedes the t test. (2013, methodology): “Dienes (2008, 2011) provided another analytical solution, with a corresponding online calculator.”
Individual Differences in Heart Rate Variability Predict the Degree of Slowing during Response Inhibition and Initiation in the Presence of Emotional Stimuli (2011, neutral): “to Dienes (2011), Rouder et al.”
Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned? (2019, methodology): “Bayes Factor For each trial we calculated a Bayes factor, following the method suggested by Dienes, Coulton and Heather (2017). This quantified how likely the data were under the null hypothesis compared to the alternative hypothesis, which was defined to be an effect size taken from a half normal d”
MorePower 6.0 for ANOVA with relational confidence intervals and Bayesian analysis (2012, methodology): “…analysis of posterior probabilities for H 0 versus H 1 is poised to emerge as a widely used alternative to NHST in psychological research (e.g., Dienes, 2011; Kruschke, 2011; Masson, 2011; Raftery, 1995; Wagenmakers, 2007; Wetzels et al., 2011; see also Dixon, 2003; Glover & Dixon, 2004, for…”