Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies

By Julian P. T. Higgins, Dan Jackson, J. Barrett, Guobing Lu, A. Ades, Ian R. White. Research Synthesis Methods, 2012

1,812 citations recorded by the source corpus.

Meta‐analyses that simultaneously compare multiple treatments (usually referred to as network meta‐analyses or mixed treatment comparisons) are becoming increasingly common. An important component of a network meta‐analysis is an assessment of the extent to which different sources of evidence are compatible, both substantively and statistically. A simple indirect comparison may be confounded if the studies involving one of the treatments of interest are fundamentally different from the studies involving the other treatment of interest. Here, we discuss methods for addressing inconsistency of evidence from comparative studies of different treatments. We define and review basic concepts of heterogeneity and inconsistency, and attempt to introduce a distinction between ‘loop inconsistency’ and ‘design inconsistency’. We then propose that the notion of design‐by‐treatment interaction provides a useful general framework for investigating inconsistency. In particular, using design‐by‐treatment interactions successfully addresses complications that arise from the presence of multi‐arm trials in an evidence network. We show how the inconsistency model proposed by Lu and Ades is a restricted version of our full design‐by‐treatment interaction model and that there may be several distinct Lu–Ades models for any particular data set. We introduce novel graphical methods for depicting networks of evidence, clearly depicting multi‐arm trials and illustrating where there is potential for inconsistency to arise. We apply various inconsistency models to data from trials of different comparisons among four smoking cessation interventions and show that models seeking to address loop inconsistency alone can run into problems. Copyright © 2012 John Wiley & Sons, Ltd.

Subfields: Experimental and Cognitive Psychology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/94bec09e6c40b49a636f9e41d4c0848be85ef396

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Statistics and Probability

Citations in the corpus

Evaluating the Quality of Evidence from a Network Meta-Analysis (2014, background): “The design-by-treatment interaction inconsistency model [29] produces a statistically significant test result for the presence of inconsistency in the network (P = 0.”
Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression (2012, methodology): “However, to illustrate interpretation as in Table 5 of the companion paper (Higgins et al., 2012), we now take ôAH11 at face value; it represents a discrepancy between the direct estimate of the H–”
CINeMA: An approach for assessing confidence in the results of a network meta-analysis (2020, methodology): “The design-by-treatment interaction test is such a global test [45,46].”
GetReal in network meta‐analysis: a review of the methodology (2016, background): “One limitation of the model is the increased complexity and the accompanying loss of power for the statistical test (Higgins et al., 2012).”
Network Meta-Analysis Using R: A Review of Currently Available Automated Packages (2014, methodology): “The presence of heterogeneity and inconsistency in the network can be quantified and assessed statistically, for which different methods have been proposed [11–16].”
Graphical Tools for Network Meta-Analysis in STATA (2013, methodology): “recently described NMA as a multivariate meta-analysis model [23,24].”
Network Meta-analysis (2015, methodology): “Statistical models for inconsistency have been proposed and can be used to assess consistency (Lu and Ades 2006; Higgins et al. 2012).”
netmeta: An R Package for Network Meta-Analysis Using Frequentist Methods (2023, positive): “Thus, for assessing the global inconsistency in a random eﬀects model, the Q B statistic can be calculated based on a full design-by-treatment interaction random eﬀects model (Higgins et al. 2012), considering the “method of moments” estimate for τ 2 (Jackson, White, and Riley 2012).”
CINeMA: Software for semiautomated assessment of the confidence in the results of network meta‐analysis (2020, positive): “CINeMA performs and displays the results of two methods; the first is a global method to assess incoherence, the design-by-treatment interaction test (26,27).”
A graphical tool for locating inconsistency in network meta-analyses (2013, methodology): “, D according to which treatments are respectively involved (see [15,16,23] for a similar approach).”