Effect Sizes for Growth-Modeling Analysis for Controlled Clinical Trials in the Same Metric as for Classical Analysis

By A. Feingold. Psychological methods, 2009

823 citations recorded by the source corpus.

The use of growth-modeling analysis (GMA)--including Hierarchical Linear Models, Latent Growth Models, and General Estimating Equations--to evaluate interventions in psychology, psychiatry, and prevention science has grown rapidly over the last decade. However, an effect size associated with the difference between the trajectories of the intervention and control groups that captures the treatment effect is rarely reported. This article first reviews two classes of formulas for effect sizes associated with classical repeated-measures designs that use the standard deviation of either change scores or raw scores for the denominator. It then broadens the scope to subsume GMA, and demonstrates that the independent groups, within-subjects, pretest-posttest control-group, and GMA designs all estimate the same effect size when the standard deviation of raw scores is uniformly used. Finally, it is shown that the correct effect size for treatment efficacy in GMA--the difference between the estimated means of the two groups at end of study (determined from the coefficient for the slope difference and length of study) divided by the baseline standard deviation--is not reported in clinical trials.

Subfields: Experimental and Cognitive Psychology, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/ebc0d199259696475c11175a3773c6fc39d748f7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Guided internet-delivered cognitive behavior therapy for post-traumatic stress disorder: A randomized controlled trial (2014, methodology): “We calculated 95% confidence intervals for traditional Cohen's d, as the sampling variance for growth modeling effect sizes has not yet been derived (Feingold, 2009).”
Recommending goals and supporting needs: An intervention to help physical education teachers communicate their expectations while supporting students' psychological needs (2019, methodology): “…affiliation goal), 8 of the 9 “condition” main effects for the ASIP + Intrinsic Goal contrast were To estimate effect sizes for these analyses, we used the independentgroups pretest-posttest design test (d IGPP-RAW) that is appropriate for repeated-measures group comparisons (Feingold, 2009).”
Preliminary Evaluation of a Tier 2 Mathematics Intervention for First-Grade Students: Using a Theory of Change to Guide Formative Evaluation Activities (2014, methodology): “As recommended by Feingold (2009), Hedges’s g was computed as the Condition-by-Time interaction effect divided by the posttest pooled standard deviation of the outcome.”
A needs‐supportive intervention to help PE teachers enhance students' prosocial behavior and diminish antisocial behavior (2018, methodology): “To estimate effect sizes, we used the independent-groups pretest-posttest design test (d IGPP-CHANGE) that is appropriate for multilevel, repeated-measures group comparisons to determine the magnitude of the change in the intervention group relative to the change in the control group on each depende”
Strengthening school readiness for Head Start children: Evaluation of a self-regulation intervention (2015, methodology): “In order to calculate effect sizes, residual variances were used to calculate standard deviations, which were then divided into the difference in means (Feingold, 2009).”
What Works in Youth Violence Prevention (2013, methodology): “In the few cases in which outcomes were assessed using growth models, effect sizes were not calculated, as methods for doing so have not been agreed upon in the field (Feingold, 2009).”
When teachers learn how to provide classroom structure in an autonomy-supportive way: Benefits to teachers and their students (2020, methodology): “…pretest-posttest design test (d IGPP-CHANGE) that is appropriate for multilevel, repeated-measures group comparisons to determine the magnitude of the change in the dependent variable in the intervention group relative to the magnitude of the change in the control group (Feingold, 2009).”
Tuning in to Teens: Improving Parent Emotion Socialization to Reduce Youth Internalizing Difficulties (2014, methodology): “Effect sizes were calculated using the recommended formula for multi-level analysis for controlled clinical trials (Feingold, 2009).”
An early numeracy intervention for first-graders at risk for mathematical learning difficulties (2021, methodology): “…effect size was calculated by dividing the difference between the estimated means of the intervention and low-control/average-control groups at the delayed posttest (determined from the coefficient for the slope difference and length of study) by the baseline standard deviation (Feingold, 2009).”
An Examination of the Efficacy of Insights in Enhancing the Academic and Behavioral Development of Children in Early Grades. (2014, positive): “Finally, effect sizes (ES) for statistically significant findings were calculated following procedures by Feingold (2009) for growth model analysis, yielding ES in the same metric as classical designs, thus facilitating comparisons across studies.”
A boost of positive affect (2013, methodology): “Also included in the table are the effect sizes for the parameters using the residual variance as the denominator (Feingold, 2009).”
Expanding Autonomy Psychological Need States From Two (Satisfaction, Frustration) to Three (Dissatisfaction): A Classroom-Based Intervention Study (2019, methodology): “…Time interaction effect, we used the independent-groups pretestposttest design test (dIGPP-RAW) that is appropriate for multilevel, repeatedmeasures group comparisons to determine the magnitude of the change in the intervention group relative to the change in the control group (Feingold, 2009).”
A Preliminary Randomized Controlled Trial of Two Social Skills Interventions for Youth With High-Functioning Autism Spectrum Disorders (2012, methodology): “For all significant effects, a partial η2 effect size was calculated for repeated-measures ANOVA models and d GMA-RAW for HLM-GMA models (Feingold, 2009).”