A review of effect sizes and their confidence intervals, Part I: The Cohen's d family

By Jean‐Christophe Goulet‐Pelletier, Denis Cousineau. The Quantitative Methods for Psychology, 2018

358 citations recorded by the source corpus.

The results of the interval estimation method popularized by Steiger and Fouladi (1997), herein called the pivoting method1, were displayed erroneously in Appendix C of our recent paper on effect sizes and corresponding confidence intervals (Goulet-Pelletier & Cousineau, 2018). This error pertains to Figure 7 of the appendix, which displayed a lower variability for the pivoting method than there really is, which gave the impression that the pivoting method was short in its coverage estimations. The error came from the fact that the lower and upper noncentrality parameters were properly found but were plotted relative to the true Cohen’s d, instead of the observed Cohen’s d. This made the error bars smaller than they should have been. We doubled-checked all the other simulations reported in our paper and this mistake only applied to the Steiger and Fouladi bars of Figure 7. We can see in the corrected figure that the pivoting method produces desirable results when the effect size is small or medium for all sample sizes. In fact, for small sample sizes, the pivoting method shows a tighter coverage with effect of small ormediummagnitude compared to the non-central method discussed in the article. However, when the effect size is large, there is a systematic underestimation of the lower bound which does not disappear with increasing sample size. In Appendix C of the published paper, we considered that the pivoting method would be less appropriate than the noncentral method in most scenarios, and more so as the true effect size increases. In light of the corrected figure, we rectify our position by saying that the pivoting method indeed returns coverage similar to the true spread of 95% of the simulated mean difference in most cases. However, we note that when examining closely the estimated target quantiles (i.e., .025 and .975 for a 95% coverage), over and above achieving a miss rate of 95%, the recommended method seems more effective in estimating the desired quantile position on average. We also note that the pivoting method is asymmetric in its coverage: the upper end of the distribution is always missed to a larger extent than the lower end. In contrast, the noncentrality method is slightly more balanced. The asymmetry of the pivoting method makes it a less precise estimator of the population quantile compared to the recommended method. Authors’ note

Subfields: Experimental and Cognitive Psychology, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/59a7d1c4e61e28a8e147ae0623b4d65ef2016d69