How Bayes factors change scientific practice

By Zoltan Dienes. 2016

361 citations recorded by the source corpus.

Bayes factors provide a symmetrical measure of evidence for one model versus another (e.g. H1 versus H0) in order to relate theory to data. These properties help solve some (but not all) of the problems underlying the credibility crisis in psychology. The symmetry of the measure of evidence means that there can be evidence for H0 just as much as for H1; or the Bayes factor may indicate insufficient evidence either way. PP-values cannot make this three-way distinction. Thus, Bayes factors indicate when the data count against a theory (and when they count for nothing); and thus they indicate when replications actually support H0 or H1 (in ways that power cannot). There is every reason to publish evidence supporting the null as going against it, because the evidence can be measured to be just as strong either way (thus the published record can be more balanced). Bayes factors can be BB-hacked but they mitigate the problem because a) they allow evidence in either direction so people will be less tempted to hack in just one direction; b) as a measure of evidence they are insensitive to the stopping rule; c) families of tests cannot be arbitrarily defined; and d) falsely implying a contrast is planned rather than post hoc becomes irrelevant (though the value of pre-registration is not mitigated).

Subfields: Artificial Intelligence, History and Philosophy of Science, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6544f1e197c8bb999331195368d5c46dbc7d639a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In History and Philosophy of Science

In Statistics and Probability

Citations in the corpus

The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective (2017, neutral): “(Dienes, 2016, p. 2) When a null hypothesis is rejected, people can mistakenly believe that the effect is solidly large, even though the results may actually allow for a large range of effect magnitudes, including very small effect sizes (e.g., Kline, 2004).”
Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future (2017, result): “This, in turn, could provide an extra argument for publishing “null” results and hence bring some balance in the published literature between “positive” and “negative” findings (see also Dienes, 2016).”
Rejecting or Accepting Parameter Values in Bayesian Estimation (2018, background): “In the psychological sciences, many proponents of BFs have routinely used 3 as the decision threshold (e.g., Dienes, 2016).”
Equivalence Tests (2017, neutral): “Second, where traditional significance test only allows researchers to reject the null hypothesis, science needs statistical approaches that allow us to conclude meaningful effects are absent (Dienes, 2016).”
When Does HARKing Hurt? Identifying When Different Types of Undisclosed Post Hoc Hypothesizing Harm Scientific Progress (2017, background): “The stringent hypothesis construction rules of parsimony and consistency with prior theory and evidence increase the prior probability that the resulting ad hoc hypotheses will be true (Dienes, 2016; Kerr, 1998; Murayama et al., 2014).”
How Do I Know What My Theory Predicts? (2019, methodology): “This claim seems to contradict Mayo (2018), who used the notion of severe testing as an argument against Bayesian statistics (contrast Vanpaemel, in press).”