Interpreting Effect Sizes of Education Interventions

By M. Kraft. 2020

905 citations recorded by the source corpus.

Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen’s standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this article, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, costs, and scalability into the process of interpreting the policy importance of effect sizes.

Subfields: Education

View Paper: https://www.semanticscholar.org/paper/87d504e273381f8d69a4f8db258c635a1ab69ddc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

Citations in the corpus

Sample Size Justification (2021, neutral): “Some researchers have suggestedtochangeCohen’sbenchmarksbasedonthedistributionofeffectsizesinaspecificfield(Bosco, Aguinis,Singh,Field,&Pierce,2015;Funder&Ozer, 2019;Hill,Bloom,Black,&Lipsey,2008;Kraft,2020; Lovakov&Agadullina,2017).”
Teacher Professional Development around the World: The Gap between Evidence and Practice (2018, background): “Comparing these effect sizes to those from a sample of 747 education-related randomized controlled trials in the United States puts them both above the 50th percentile in terms of effectiveness (Kraft 2020).”
Beyond small, medium, or large: points of consideration when interpreting effect sizes (2019, methodology): “Kraft (2019) proposes a new approach for causal studies on achievement, a matrix including cost and scalability.”
Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2020, background): “These characteristics make wise interventions particularly suitable for scaling up, even where the average size of their effect on the overall population is relatively small (Bakker et al., 2019; Kraft, 2020; Slavin & Madden, 2011).”
The Effect of Principal Behaviors on Student, Teacher, and School Outcomes: A Systematic Review and Meta-Analysis of the Empirical Literature (2019, methodology): “Relying heavily on the clear analytic approach of Kraft, Blazar, and Hogan (2018), we address the clustered nature of our outcomes by estimating all models using robust variance estimation methods (Hedges, Tipton, & Johnson, 2010; Tanner-Smith & Tipton, 2014), which account both for the different pr”
A Meta-Analytic Review of the Relations Between Motivation and Reading Achievement for K–12 Students (2020, background): “However, the education research literature has since provided ample evidence to inform a greater basis for interpretation (see Kraft, 2019, for review).”
Education in Africa: What Are We Learning? (2020, background): “5 Two earlier examples include Kremer and Miguel (2007) and Baird et al. (2011). education interventions (Kraft, 2020; Evans and Yuan 2020).”
What Can Be Learned from Growth Mindset Controversies? (2020, background): “A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript still used in our field, have led the field astray (also see a discussion in Kraft, 2020).”