A re-evaluation of random-effects meta-analysis

By Julian P. T. Higgins, S. Thompson, David J. Spiegelhalter. Journal of the Royal Statistical Society. Series A,, 2009

2,239 citations recorded by the source corpus.

Summary. Meta‐analysis in the presence of unexplained heterogeneity is frequently undertaken by using a random‐effects model, in which the effects underlying different studies are assumed to be drawn from a normal distribution. Here we discuss the justification and interpretation of such models, by addressing in turn the aims of estimation, prediction and hypothesis testing. A particular issue that we consider is the distinction between inference on the mean of the random‐effects distribution and inference on the whole distribution. We suggest that random‐effects meta‐analyses as currently conducted often fail to provide the key results, and we investigate the extent to which distribution‐free, classical and Bayesian approaches can provide satisfactory methods. We conclude that the Bayesian approach has the advantage of naturally allowing for full uncertainty, especially for prediction. However, it is not without problems, including computational intensity and sensitivity to a priori judgements. We propose a simple prediction interval for classical meta‐analysis and offer extensions to standard practice of Bayesian meta‐analysis, making use of an example of studies of ‘set shifting’ ability in people with eating disorders.

Subfields: Food Science, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/b42717775886e436d97964fca187c3248684112e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Food Science

In Statistics and Probability

Citations in the corpus

General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi‐trait models for continuous and categorical characters (2010, methodology): “This type of assumption has been criticized even in the context of controlled clinical trials (Higgins et al., 2009), and would seem untenable when the data are associated with different species (West & Sheldon, 2002).”
Visualizing Assumptions and Results in Network Meta-analysis: The Network Graphs Package (2015, background): “The extent of uncertainty in the estimated treatment eﬀects in meta-analysis is re-ﬂected not only by the conﬁdence intervals but also by the predictive intervals that incorporate the extent of heterogeneity (Higgins, Thompson, and Spiegelhalter 2009; Riley, Higgins, and Deeks 2011).”
The heterogeneity statistic I2 can be biased in small meta-analyses (2015, methodology): “Under a random-effects model, by contrast, we assume that the true effects βk in the meta-analysis were drawn at random from a larger population of effects, and we seek to make inferences about that larger population [16].”
Network Meta-analysis (2015, neutral): “…ranks. cumulative makes the bar or line graph show cumulative ranks. predict ranks the true eﬀects in a future study with the same covariates, thus allowing for heterogeneity as well as parameter uncertainty, as in the calculation of prediction intervals (Higgins, Thompson, and Spiegelhalter 2009).”
Replication in genome-wide association studies (2009, background): “where t k−2 is the 100(1 − α/2)% percentile of the t-distribution with k − 2 degrees of freedom [20].”
Conceptual and Technical Challenges in Network Meta-analysis (2013, neutral): “Random-effects meta-analysis can be applied (33, 34).”
Meta-analysis of individual participant data: rationale, conduct, and reporting (2010, result): “The one step and two step individual participant data meta-analyses gave identical results. The pooled treatment effect was estimated at -10.16 (95% CI -12.27 to -8.06), indicating that hypertension treatment reduced systolic blood pressure by, on average, 10.16 mm Hg compared with controls. There w”
Introduction, comparison, and validation of Meta‐Essentials: A free and simple tool for meta‐analysis (2017, background): “139 If we assume that all studies provide estimates of different true effects, we must also assume that no single overall effect size can express these different true effects' best.(39) Therefore, the prediction interval accurately embraces the notion of heterogeneity and the dispersion of true effe”
Plea for routinely presenting prediction intervals in meta-analysis (2016, background): “effects are to be expected for 95% of similar (exchangeable) studies that might be conducted in the future.(4) Therefore, it is well suited to evaluate the variability of the effect of an intervention over different settings.”
A re‐evaluation of fixed effect(s) meta‐analysis (2018, background): “The random-effects model is discussed at length elsewhere (Higgins et al., 2009); we particularly note here that its assumptions are, in general, different from both the common effect and fixed effects model.”
Effects of Payment for Environmental Services (PES) on Deforestation and Poverty in Low and Middle Income Countries: A Systematic Review (2014, methodology): “…the empirical Bayes procedure, the confidence interval for the random effects mean applies the small sample adjustment of Knapp and Hartung (2003), and the predictive interval uses a t-distribution with degrees of freedom equal to the number of studies minus 1, following Higgins et al. (2009).”
Multivariate meta-analysis: Potential and promise (2011, background): “Similarly, it allows a joint prediction region [63] for the true sensitivity and specificity in an individual study setting.”