A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses

By Dean Langan, Julian P. T. Higgins, Dan Jackson, J. Bowden, A. Veroniki, Evangelos Kontopantelis, W. Viechtbauer, M. Simmonds. Research Synthesis Methods, 2018

871 citations recorded by the source corpus.

Studies combined in a meta‐analysis often have differences in their design and conduct that can lead to heterogeneous results. A random‐effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian‐Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta‐analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian‐Laird estimator is negatively biased in scenarios with small studies and in scenarios with a rare binary outcome. Results also show the Paule‐Mandel method has considerable positive bias in meta‐analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta‐analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta‐analysis. The estimated summary effect of the meta‐analysis and its confidence interval derived from the Hartung‐Knapp‐Sidik‐Jonkman method are more robust to changes in the heterogeneity variance estimate and show minimal deviation from the nominal coverage of 95% under most of our simulated scenarios.

Subfields: Economics and Econometrics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/264ababb39dc4b8737425a30c9fb75c5d8eb2bf4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Andrea Benedetti
2. R. Steele
3. S. McGrath
4. Xiaofei Zhao
5. B. Thombs

Citations in the corpus

• Estimating the sample mean and standard deviation from commonly reported quantiles in meta-analysis (2019, methodology): “The restricted maximum likelihood method was used to estimate heterogeneity in all meta-analyses.(30) The QE and BC methods were less biased for estimating the pooled mean compared to the 2528 Statistical Methods in Medical Research 29(9)”