Evaluating Effect Size in Psychological Research: Sense and Nonsense

By D. Funder, D. Ozer. Advances in Methods and Practices in Psychological Science, 2019

2,817 citations recorded by the source corpus.

Effect sizes are underappreciated and often misinterpreted—the most common mistakes being to describe them in ways that are uninformative (e.g., using arbitrary standards) or misleading (e.g., squaring effect-size rs). We propose that effect sizes can be usefully evaluated by comparing them with well-understood benchmarks or by considering them in terms of concrete consequences. In that light, we conclude that when reliably estimated (a critical consideration), an effect-size r of .05 indicates an effect that is very small for the explanation of single events but potentially consequential in the not-very-long run, an effect-size r of .10 indicates an effect that is still small at the level of single events but potentially more ultimately consequential, an effect-size r of .20 indicates a medium effect that is of some explanatory and practical use even in the short run and therefore even more important, and an effect-size r of .30 indicates a large effect that is potentially powerful in both the short and the long run. A very large effect size (r = .40 or greater) in the context of psychological research is likely to be a gross overestimate that will rarely be found in a large sample or in a replication. Our goal is to help advance the treatment of effect sizes so that rather than being numbers that are ignored, reported without interpretation, or interpreted superficially or incorrectly, they become aspects of research reports that can better inform the application and theoretical development of psychological research.

View Paper: https://www.semanticscholar.org/paper/d60c9e809219808908ab47e7c2061744cab1721b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Countering Misinformation and Fake News Through Inoculation and Prebunking (2021, background): “…under 7.6% (Epstein & Robertson, 2015) and the 2016 election was decided by razor-thin margins in a few swing states), these effects can be considered meaningful when scaled (Funder & Ozer, 2019) and commensurate with effect sizes in persuasion research (Banas & Rains, 2010; Walter & Murphy, 2018).”
Family Business Succession: What’s Motivation Got to Do With It? (2019, result): “Even though our effect sizes were relatively small (Funder & Ozer, 2019), the relation we found between successor intrinsic motivation and succession outcomes roughly translates into a 7% improvement in succession outcomes.”
Influencing Climate Change Attitudes in the United States: A Systematic Review and Meta-Analysis (2020, background): “In contrast, effects could accumulate over time or after repeated exposure and eventually have a meaningful impact (Funder & Ozer, 2019).”
Loneliness and the Big Five Personality Traits: A Meta–Analysis (2020, background): “It has been argued that the conventions for effect size interpretation proposed by Cohen (1992) are not appropriate for individual differences research (Gignac & Szodorai, 2016; Funder & Ozer, 2019).”
An Agenda for Open Science in Communication (2020, background): “Second, they overestimate the size of those effects that are found (Funder & Ozer, 2019).”
The Role of Personality in COVID-19-Related Perceptions, Evaluations, and Behaviors: Findings Across Five Samples, Nine Traits, and 17 Criteria (2021, result): “…observed effect sizes (of the consistent patterns) are much in line with effect sizes typically found in personality and social psychology research (Funder & Ozer, 2019; Gignac & Szodorai, 2016), further supporting the interpretation that personality traits do not only generally matter for a broad…”
Self-regulation in childhood as a predictor of future outcomes: A meta-analytic review. (2020, background): “…events (but potentially more consequential in the long run), an effect-size r of .20 was considered medium and of some explanatory and practical use even in the short run, and an effect-size r of .30 was considered large and potentially powerful in both the short and long run (Funder & Ozer, 2019).”
Functional Fear Predicts Public Health Compliance in the COVID-19 Pandemic (2020, methodology): “For example, Funder and Ozer (2019) reviewed various summative analyses of the psychology and reported r = .20 as the typical effect size.”
A Meta-Analysis of the Relationship Between Sleep Problems and Loneliness (2020, methodology): “We emphasize that although these effect-size cutoffs are commonly used in psychological research, as we note in our Discussion section, they are limited in their interpretability (Funder & Ozer, 2019).”