The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method

By J. Inthout, John P. A. Ioannidis, G. Borm. BMC Medical Research Methodology, 2014

1,885 citations recorded by the source corpus.

BackgroundThe DerSimonian and Laird approach (DL) is widely used for random effects meta-analysis, but this often results in inappropriate type I error rates. The method described by Hartung, Knapp, Sidik and Jonkman (HKSJ) is known to perform better when trials of similar size are combined. However evidence in realistic situations, where one trial might be much larger than the other trials, is lacking. We aimed to evaluate the relative performance of the DL and HKSJ methods when studies of different sizes are combined and to develop a simple method to convert DL results to HKSJ results.MethodsWe evaluated the performance of the HKSJ versus DL approach in simulated meta-analyses of 2–20 trials with varying sample sizes and between-study heterogeneity, and allowing trials to have various sizes, e.g. 25% of the trials being 10-times larger than the smaller trials. We also compared the number of “positive” (statistically significant at p < 0.05) findings using empirical data of recent meta-analyses with > = 3 studies of interventions from the Cochrane Database of Systematic Reviews.ResultsThe simulations showed that the HKSJ method consistently resulted in more adequate error rates than the DL method. When the significance level was 5%, the HKSJ error rates at most doubled, whereas for DL they could be over 30%. DL, and, far less so, HKSJ had more inflated error rates when the combined studies had unequal sizes and between-study heterogeneity. The empirical data from 689 meta-analyses showed that 25.1% of the significant findings for the DL method were non-significant with the HKSJ method. DL results can be easily converted into HKSJ results.ConclusionsOur simulations showed that the HKSJ method consistently results in more adequate error rates than the DL method, especially when the number of studies is small, and can easily be applied routinely in meta-analyses. Even with the HKSJ method, extra caution is needed when there are = <5 studies of very unequal sizes.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8c392519ae6638e4fc827796781823e0cefdce09

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses (2018, result): “These findings agree with a previous 533 simulation study (IntHout et al., 2014), in which the HKSJ method was compared with other 534 confidence interval methods for both continuous and binary outcome measures.”
Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ (2016, background): “Indeed, there is much ongoing debate in the meta-analysis literature about the best method to estimate τ(2) and (subsequently) θ [22,23], and we return to this issue in Section 3.”
Plea for routinely presenting prediction intervals in meta-analysis (2016, result): “The difference may be caused by publication bias, the higher number of primary studies in their sample (median 9 vs 4 in our set(15)) and by their use of the DerSimonian-Laird approach which can result in too many statistically significant findings, whereas we used the HKSJ approach.(16) However, re”
Meta-Analysis in Clinical Trials Revisited (2015, methodology): “study are small, or highly unequal [25].”
Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods (2014, methodology): “However, the HartungKnapp-Sidik-Jonkman random-effects model assuming a t-distribution of log (RR) (for dichotomous outcomes) seems to be a more valid meta-analysis method [22].”
Opioids for Chronic Noncancer Pain: A Systematic Review and Meta-analysis (2018, background): “Current systematic reviews have a number of limitations, leaving uncertainty with regard to the benefits and harms associated with opioid therapy for CNCP.”
Mid and long-term neurological and neuropsychiatric manifestations of post-COVID-19 syndrome: A meta-analysis (2022, neutral): “SidikJonkman estimator was used for tau [30], Hartung-Knapp adjustment for Confidence intervals (CI) [31], and the Freeman-Tukey double arcsine transformation to calculate prevalence rates for each outcome.”