Equivalence Tests

By Daniël Lakens. Social Psychology and Personality Science, 2017

1,398 citations recorded by the source corpus.

Scientists should be able to provide support for the absence of a meaningful effect. Currently, researchers often incorrectly conclude an effect is absent based a nonsignificant result. A widely recommended approach within a frequentist framework is to test for equivalence. In equivalence tests, such as the two one-sided tests (TOST) procedure discussed in this article, an upper and lower equivalence bound is specified based on the smallest effect size of interest. The TOST procedure can be used to statistically reject the presence of effects large enough to be considered worthwhile. This practical primer with accompanying spreadsheet and R package enables psychologists to easily perform equivalence tests (and power analyses) by setting equivalence bounds based on standardized effect sizes and provides recommendations to prespecify equivalence bounds. Extending your statistical tool kit with equivalence tests is an easy way to improve your statistical and theoretical inferences.

Subfields: Computer Networks and Communications, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3590ee5cd78891e1cdc95429ba3bedd0ae433704

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Statistics and Probability

Citations in the corpus

Deaf children need language, not (just) speech (2019, neutral): “The lack of a statistically significant difference does not constitute evidence for equivalence (Lakens, 2017).”
Using Anchor-Based Methods to Determine the Smallest Effect Size of Interest (2019, methodology): “…interest (SESOI) in a-priori power analyses, or as the boundaries for an equivalence range when performing an equivalence or minimal effect tests (Lakens, 2014, 2017), for studies that use the PANAS in the population our samples were drawn from (i.e., Dutch university students, and UK…”
Equivalence Testing for Psychological Research: A Tutorial (2018, methodology): “Citation for published version (APA): Lakens, D., Scheel, A. M., & Isager, P. (2018). Equivalence testing for psychological research: a tutorial.”
Indices of Effect Existence and Significance in the Bayesian Framework (2019, background): “…the null hypothesis from the classic point-null hypothesis to a range of values considered negligible or too small to be of any practical relevance (the Region of Practical Equivalence – ROPE; Kruschke, 2014; Lakens, 2017; Lakens et al., 2018), usually spread equally around 0 (e.g., [−0.1; 0.1]).”
When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias (2018, background): “For example, if researchers decide their SESOI is a medium e ﬀ ect size of η 2 = .0588 a study with 87 participants in each of two groups will in the long run have a power of 0.9 to detect the SESOI, or reject it in an equivalence test (Lakens, 2017).”
The Replicability Crisis and Public Trust in Psychological Science (2018, methodology): “Using the TOSTER package in R (Lakens, 2017) we calculated that for alpha = .0307, and 90% power to exclude an effect size outside the equivalence range of d = −0.3 and d = 0.3, we require a sample size of 275 per condition (Lakens, Scheel, & Isager, 2018).”
Effects of Public Green Space on Acute Psychophysiological Stress Response: A Systematic Review and Meta-Analysis of the Experimental and Quasi-Experimental Evidence (2019, methodology): “When meta-analyses did not indicate significant differences in effects, we used the two one-sided test (TOST) procedure to assess the evidence for statistical equivalence (Lakens, 2017).”