Measuring inconsistency in meta-analyses
By Julian P. T. Higgins, S. Thompson, J. Deeks, Douglas G. Altman. British medical journal, 2003
57,483 citations recorded by the source corpus.
Cochrane Reviews have recently started including the quantity I 2 to help readers assess the consistency of the results of studies in meta-analyses. What does this new quantity mean, and why is assessment of heterogeneity so important to clinical practice? Systematic reviews and meta-analyses can provide convincing and reliable evidence relevant to many aspects of medicine and health care.1 Their value is especially clear when the results of the studies they include show clinically important effects of similar magnitude. However, the conclusions are less clear when the included studies have differing results. In an attempt to establish whether studies are consistent, reports of meta-analyses commonly present a statistical test of heterogeneity. The test seeks to determine whether there are genuine differences underlying the results of the studies (heterogeneity), or whether the variation in findings is compatible with chance alone (homogeneity). However, the test is susceptible to the number of trials included in the meta-analysis. We have developed a new quantity, I 2, which we believe gives a better measure of the consistency between trials in a meta-analysis. Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis. Indeed, several hierarchical systems for grading evidence state that the results of studies must be consistent or homogeneous to obtain the highest grading.2–4 Tests for heterogeneity are commonly used to decide on methods for combining studies and for concluding consistency or inconsistency of findings.5 6 But what does the test achieve in practice, and how should the resulting P values be interpreted? A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. The usual test statistic …
Subfields: Critical Care and Intensive Care Medicine, General Health Professions, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/bef40c4e1347f940e8f3d420e24d9a070863e8d9
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Critical Care and Intensive Care Medicine
In General Health Professions
- David Moher
- Douglas G. Altman
- Georgia Salanti
- John P. A. Ioannidis
- P. Erwin
- D. A. Cook
- Mike Clarke
- Julian P. T. Higgins
In Statistics and Probability
- John P. A. Ioannidis
- David Moher
- Douglas G. Altman
- Julian P. T. Higgins
- Georgia Salanti
- Ian R. White
- R. Riley
- Haitao Chu
Citations in the corpus
- Activity Achievement Emotions and Academic Performance: A Meta-analysis (2021, background): “Higgins et al. (2003) suggests benchmarks of low, moderate, and high to values of 25%, 50%, 75% for I2, which we apply here, and examined effects of moderators wherever I2 was above 25%.”
- A systematic review and meta-analysis of dietary patterns and depression in community-dwelling adults. (2014, methodology): “Heterogeneity was assessed by using the I(2) statistic (14).”
- Self-regulation in childhood as a predictor of future outcomes: A meta-analytic review. (2020, background): “Values of 25%, 50% and 75% are considered to represent low, medium and high levels of heterogeneity (Higgins et al., 2003).”
- Meta‐analysis of variation: ecological and evolutionary applications and beyond (2015, methodology): “Moreover, we observed large heterogeneity (sensu Higgins et al. 2003) in the analysis of both metrics (REMA: I2 for d = 89 36%, I2 for lnRR = 94 64%; MLMA: I2 for d = 90 78%, I2 for lnRR = 96 51%; see also Table S2, for the details of I2 for MLMA as described in Nakagawa & Santos 2012).”
- The effects of mindfulness-based stress reduction therapy on mental health of adults with a chronic medical disease: a meta-analysis. (2010, background): “A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity [23].”