Revised standards for statistical evidence

By Valen E. Johnson. Proceedings of the National Academy of Sciences of the United States of America, 2013

717 citations recorded by the source corpus.

Subfields: Economics and Econometrics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/58cd525db8e3eba143dc0b1a71df86d3cea66fc2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Statistics and Probability

Citations in the corpus

How to Choose the Level of Significance: A Pedagogical Note (2019, positive): “Based on this, Johnson (2013) recommends adoption of the “revised standards for statistical evidence” by setting the level of significance at 0.005 or 0.001, instead of 0.05 and 0.01 (as in the example in Figure 1).”
Statistical reform in information retrieval? (2014, background): “Another alternative is the Bayesian approach to hypothesis testing [2, 10, 11], which was first conceptualised in 1935.”
Harold Jeffreys’s default Bayes factor hypothesis tests: Explanation, extension, and application in psychology (2016, background): “This remark is still relevant because data for which p-values are larger1 than .01 (i.e., “p < .05, reject the null hypothesis”) generally do not pass this threshold of2 evidence (Berger & Delampady, 1987; Edwards, Lindman, & Savage, 1963; Johnson, 2013”
Bayesian inference for psychology. Part II: Example applications with JASP (2017, background): “05 boundary (“reject the null hypothesis”) yield evidence that is weaker than BF10 = 3 (e.g., Berger & Delampady 1987; Edwards, Lindman, & Savage 1963; Johnson, 2013; Wagenmakers et al., 2017; Wetzels et al., 2011).”
Three Recommendations for Improving the Use of p-Values (2019, methodology): “Fortunately, several methods (Edwards, Lindman, and Savage 1963; Vovk 1993; Johnson 2013; Sellke, Bayarri, and Berger 2001) have been developed for using the p-value to calculate an upper bound on BF, called the Bayes factor bound, which we denote by BFB.”
Abandon Statistical Significance (2017, background): “Speciﬁcally, the uniformly most powerful Bayesian tests (UMPBTs) that underlie the proposal were introduced and defended by Johnson (2013b) in parallel with his call in Johnson (2013a)—and now repeated in Benjamin et al. (2018)—to use 0.005 as the new threshold.”
Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests (2018, background): “In the aspect of reproducibility concerns of scientific research, Johnson (2013) traced one major cause of nonreproducibility as the routine use of the null hypothesis testing procedure.”
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results (2018, background): “Similar subjective and continually debated decisions have had to be made about the cut-off for statistical significance (Benjamin et al., 2017; Johnson, 2013).”
Testing hypotheses via a mixture estimation model (2014, background): “X Finally, there is a temptation to create pseudo-frequentist equivalents such as q-values (Johnson and Rossell, 2010; Johnson, 2013a,b) with even less Bayesian justification.”
FUNDAMENTAL LIMITATIONS OF HIGH CONTRAST IMAGING SET BY SMALL SAMPLE STATISTICS (2014, background): “We do not address the choice of statistical paradigm between the frequentist and bayesian approaches (see Johnson (2013) for an interesting review of the subject).”
What Are the Odds? A Practical Guide to Computing and Reporting Bayes Factors (2014, methodology): “It is only as p-values approach .01 that evidence starts becoming substantial, according to the calculated Bayes factors (Johnson, 2013; Nuzzo, 2014).”
Sample Size Justification (2021, neutral): “In other fields such attempts have been unsuccessful (e.g., Johnson (2013)).”
An Introduction to Second-Generation p-Values (2019, background): “…to alter the daily practice of statistics in applied disciplines (e.g., Cristea and Ioannidis 2018). p-Value “improvements” have been limited to revising the threshold for significance (Johnson 2013; Benjamin et al. 2018; Lakens et al. 2018) and tweaking the interpretation (Ioannidis 2018).”