Three Recommendations for Improving the Use of p-Values
By Daniel J. Benjamin, James O. Berger. American Statistician, 2019
216 citations recorded by the source corpus.
ABSTRACT Researchers commonly use p-values to answer the question: How strongly does the evidence favor the alternative hypothesis relative to the null hypothesis? p-Values themselves do not directly answer this question and are often misinterpreted in ways that lead to overstating the evidence against the null hypothesis. Even in the “post p < 0.05 era,” however, it is quite possible that p-values will continue to be widely reported and used to assess the strength of evidence (if for no other reason than the widespread availability and use of statistical software that routinely produces p-values and thereby implicitly advocates for their use). If so, the potential for misinterpretation will persist. In this article, we recommend three practices that would help researchers more accurately interpret p-values. Each of the three recommended practices involves interpreting p-values in light of their corresponding “Bayes factor bound,” which is the largest odds in favor of the alternative hypothesis relative to the null hypothesis that is consistent with the observed data. The Bayes factor bound generally indicates that a given p-value provides weaker evidence against the null hypothesis than typically assumed. We therefore believe that our recommendations can guard against some of the most harmful p-value misinterpretations. In research communities that are deeply attached to reliance on “p < 0.05,” our recommendations will serve as initial steps away from this attachment. We emphasize that our recommendations are intended merely as initial, temporary steps and that many further steps will need to be taken to reach the ultimate destination: a holistic interpretation of statistical evidence that fully conforms to the principles laid out in the ASA statement on statistical significance and p-values.
Subfields: Artificial Intelligence, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/57293d5072af8856f392f29475669ebb5d18cb2a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Statistics and Probability
Citations in the corpus
- Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests (2018, methodology): “Focusing on discussions surrounding p-values, a special issue of The American Statistician (2019) contains many proposals to adjust, abandon or provide alternatives to p-value (e.g., Benjamin and Berger 2019; Betensky 2019; Billheimer 2019; Manski 2019; Matthews 2019, among others).”
- E-values: Calibration, combination and applications (2019, unknown): “Since is used (see, e.g., Benjamin and Berger [2019], Recommendations 2 and 3); we will refer to it as the VS bound (abbreviating “Vovk–Sellke bound”, as used in, e.g., the JASP package).”
- Enhancing Rigor in Quantitative Entrepreneurship Research (2019, background): “…between two compared groups) would be equal to or more extreme than its observed value” (Wasserstein & Lazar, 2016), but it is often misinterpreted (Benjamin & Berger, 2019; Greenland et al., 2016; McShane et al., 2019; Wasserstein et al., 2019).7 As defined by Wasserstein et al. (2019), while…”
- Moving to a World Beyond “p < 0.05” (2019, unknown): “Betensky (2019) also reminds us to interpret the p -value in the context of sample size and meaningful effect size.”