The heterogeneity statistic I 2 can be biased in small meta-analyses

By Paul T. von Hippel. 2015

576 citations recorded by the source corpus.

Background Estimated effects vary across studies, partly because of random sampling error and partly because of heterogeneity. In meta-analysis, the fraction of variance that is due to heterogeneity is estimated by the statistic I2. We calculate the bias of I2, focusing on the situation where the number of studies in the meta-analysis is small. Small meta-analyses are common; in the Cochrane Library, the median number of studies per meta-analysis is 7 or fewer.

Subfields: Endocrinology, Diabetes and Metabolism, Neurology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/a5b4b40fe8e2d06f8113378afd3a66e19b8427b1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Endocrinology, Diabetes and Metabolism

In Neurology

In Statistics and Probability

Citations in the corpus

The Relationship between Visuospatial Working Memory and Mathematical Performance in School-Aged Children: a Systematic Review (2019, background): “Following findings by Von Hippel (2015), it is important to consider the I2 statistic in relation to the number of studies included in the meta-analysis; however, 35 studies should be sufficient to mitigate the potential for bias with a small number of studies.”
Brain volume and intelligence: The moderating role of intelligence measurement quality (2017, background): “However, given Cochran's Q is substantially affected by statistical power (von Hippel, 2015), emphasis was placed on the interpretation of I2, the proportion of the variance in the correlations that was due to heterogeneity.”
Meta-analysis of the relationship between problem gambling, excessive gaming and loot box spending (2021, methodology): “…employed a Random Effects, rather than Fixed Effects Model which does not assume an identical effect/relationship across the data and distribution, and performs better at identifying the average effect size when heterogeneity is high (Hedges & Vevea, 1998; Higgins et al., 2019; Von Hippel, 2015).”