Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods.

By Andy Field. Psychological methods, 2001

733 citations recorded by the source corpus.

The efficacy of the Hedges and colleagues, Rosenthal-Rubin, and Hunter-Schmidt methods for combining correlation coefficients was tested for cases in which population effect sizes were both fixed and variable. After a brief tutorial on these meta-analytic methods, the author presents two Monte Carlo simulations that compare these methods for cases in which the number of studies in the meta-analysis and the average sample size of studies were varied. In the fixed case the methods produced comparable estimates of the average effect size; however, the Hunter-Schmidt method failed to control the Type I error rate for the associated significance tests. In the variable case, for both the Hedges and colleagues and Hunter-Schmidt methods, Type I error rates were not controlled for meta-analyses including 15 or fewer studies and the probability of detecting small effects was less than .3. Some practical recommendations are made about the use of meta-analysis.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/98ce8a12bf1f4a9626e04fdcc2e26c39aa710de3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Robotics as Means to Increase Achievement Scores in an Informal Learning Environment (2007, background): “A Pearson's correlation coefficient as a measure of effect can lie between 0 (no effect) to 1 (a perfect effect) (Field, 2001).”
Convergent effects of elevation on functional leaf traits within and among species (2014, methodology): “We transformed effect sizes using Fisher’s r-to-z transformation, z = tanh 1r, to ensure that the distribution of effect sizes approximated a normal distribution (DeCoster 2004), and conducted a random-effects meta-analysis on the transformed correlation coefficients (Field 2001).”
A meta-analysis of burnout with job demands, resources, and attitudes (2011, background): “All rights reserved. doi:10.1016/j.jvb.2011.03.007 Contents lists available at ScienceDirect Journal of Vocational Behavior j ourna l homepage: www.e lsev ie r.com/ locate / jvb analysis (Field, 2001).”
A meta-analysis of the technology acceptance model (2006, result): “In general, results for the three methods are similar [23] [81].”
Understanding the Role and Methods of Meta-Analysis in IS Research (2005, methodology): “Under a fixed-effects model, the Rosenthal-Rubin method employs essentially the same techniques as the Hedges-Olkin method, except for the significance test of combined effect size [Field, 2001].”
Does Team Training Improve Team Performance? A Meta-Analysis (2008, background): “Field (2001) noted that the random effects model is considered to be “more realistic than the fixed-effect model on the majority of occasions” (p. 162).”
Divestiture and Firm Performance: A Meta-Analysis (2010, methodology): “Apart from cumulating research findings across studies, meta-analysis also allows us to contribute to the theoretical development of the field by identifying moderator effects that may be hidden in the empirical findings. Our analysis leads us to posit that this may be especially so with respect to ”
Fixed- versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results. (2009, background): “FE models consider only the first source of sampling error and do not take into account the second source (Field, 2001, 2003, 2005; Hedges & Vevea, 1998; Hunter & Schmidt, 2000; Overton, 1998; Raudenbush, 1994; Schulze, 2004).”
Psychometric Properties of the Parent and Teacher Versions of the Strengths and Difficulties Questionnaire for 4- to 12-Year-Olds: A Review (2010, methodology): “The weighted mean Z-score was transformed back to a correlation coefficient r (Field 2001).”
Consumer Acceptance and Use of Information Technology: A Meta-Analytic Evaluation of UTAUT2 (2020, methodology): “Furthermore, random effect model is considered more realistic as it allows the researcher to make generalization about the research domain as a whole without restricting the findings only to studies involved in meta-analysis (Field 2001).”
Uncertainty in the workplace: Examining role ambiguity and role conflict, and their link to depression—a meta-analysis (2014, background): “A number of studies have investigated the various impacts of new situations on the well-being of employees (eg, Ashford, 1988 ; Begley & Czajka, 1993 ; Ferrie, Shipley, Marmot, Stansfeld, & Smith, 1998 ).”
A Reliability Generalization Study of the Marlowe-Crowne Social Desirability Scale (2002, methodology): “The sampling distribution of correlation estimates (for estimates not based on large samples) has been found to be skewed (e.g., Field, 2001).”
Assessing heterogeneity in meta-analysis: Q statistic or I2 index? (2006, methodology): “, 1996 ; Field , 2001 ; Hedges & Vevea , 1998 ; Overton , 1998 ) .”