Confidence Intervals for Standardized Effect Sizes: Theory, Application, and Implementation

By Ken Kelley. 2007

249 citations recorded by the source corpus.

The behavioral, educational, and social sciences are undergoing a paradigmatic shift in methodology, from disciplines that focus on the dichotomous outcome of null hypothesis significance tests to disciplines that report and interpret effect sizes and their corresponding confidence intervals. Due to the arbitrariness of many measurement instruments used in the behavioral, educational, and social sciences, some of the most widely reported effect sizes are standardized. Although forming confidence intervals for standardized effect sizes can be very beneficial, such confidence interval procedures are generally difficult to implement because they depend on noncentral t, F, and x2 distributions. At present, no main-stream statistical package provides exact confidence intervals for standardized effects without the use of specialized programming scripts. Methods for the Behavioral, Educational, and Social Sciences (MBESS) is an R package that has routines for calculating confidence intervals for noncentral t, F, and x2 distributions, which are then used in the calculation of exact confidence intervals for standardized effect sizes by using the confidence interval transformation and inversion principles. The present article discusses the way in which confidence intervals are formed for standardized effect sizes and illustrates how such confidence intervals can be easily formed using MBESS in R.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/83e964924f8329100ae3f842b86f038ebb4ace52

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Replication, statistical consistency, and publication bias. (2013, methodology): “Figure 7 plots the estimated e↵ect size for each experiment and its 95% confidence interval (Kelley, 2007).”
Personality, Punishment and Public Goods: Strategic Shifts towards Cooperation as a Matter of Dispositional Honesty–Humility (2012, methodology): “26: 245–254 (2012) DOI: 10.1002/per using the procedures supplied by Kelley (2007) revealed a lower bound of .01; one the one hand, this demonstrates that the effect in question can be considered greater than zero within a conventional level of confidence.”
Alternatives to P value: confidence interval and effect size (2016, methodology): “This unique distribution may be converted into a normal distribution by Fisher’s r-to-z transformation [18].”
Retrieval Practice Consistently Benefits Student Learning: a Systematic Review of Applied Research in Schools and Classrooms (2019, methodology): “To compute the effect sizes and 95% confidence intervals around them, we used the MBESS package for R (functions ci.sm and ci.smd for within- and betweensubject designs, respectively; Kelley 2007a, 2007b, 2017).”
A primer on investigating the after effects of acute bouts of physical activity on cognition (2019, methodology): “1 1s 1 2 1 2 Whereas the confidence interval for the repeated measures Cohen’s d comparison (Kelley, 2007; Nakagawa & Cuthill, 2007) is calculated as: ⎜ ⎟ ⎛ ⎝ − ≤ ≤ − ⎞ ⎠ ncp Lower Limit r n d ncp Upper Limit r n $ .”
Sample Size Justification (2021, unknown): “The MBESS package in R provides functions to compute sample sizes for a wide range of tests (Kelley, 2007).”
Effect size measures for mediation models: quantitative strategies for communicating indirect effects. (2011, unknown): “…confidence intervals for effect sizes because the real interest lies not in the estimated value but in the population value (Balluerka, Gómez, & Hidalgo, 2005; Bird, 2002; Cumming & Finch, 2001; Fidler & Thompson, 2001; Henson, 2006; Kelley, 2007a; Kirk, 1996; Smithson, 2001; Thompson, 2002, 2007).”
Computation of measures of effect size for neuroscience data sets (2011, methodology): “Software that does provide such functionality, e.g. scripts or packagesdesignedforthefreely availablestatistics program‘ r ’(Kelley, 2007; Nakagawa & Cuthill, 2007) or standalone programs (see pointers in Kline, 2004; Jordan et al. , 2010), seems not to be widely known or used in the neurosciences.”