Bias modelling in evidence synthesis

By Rebecca M. Turner, David J. Spiegelhalter, Gordon C. S. Smith, S. Thompson. Journal of the Royal Statistical Society. Series A,, 2009

319 citations recorded by the source corpus.

Summary. Policy decisions often require synthesis of evidence from multiple sources, and the source studies typically vary in rigour and in relevance to the target question. We present simple methods of allowing for differences in rigour (or lack of internal bias) and relevance (or lack of external bias) in evidence synthesis. The methods are developed in the context of reanalysing a UK National Institute for Clinical Excellence technology appraisal in antenatal care, which includes eight comparative studies. Many were historically controlled, only one was a randomized trial and doses, populations and outcomes varied between studies and differed from the target UK setting. Using elicited opinion, we construct prior distributions to represent the biases in each study and perform a bias‐adjusted meta‐analysis. Adjustment had the effect of shifting the combined estimate away from the null by approximately 10%, and the variance of the combined estimate was almost tripled. Our generic bias modelling approach allows decisions to be based on all available evidence, with less rigorous or less relevant studies downweighted by using computationally simple methods.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/2c4df64556d3210bb40e8d36e869d8210e23aa7d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Evidence Synthesis for Decision Making 4 (2013, background): “The place for an enquiry into consistency is alongside a consideration of heterogeneity and its causes and, where appropriate, the reduction of heterogeneity through covariate adjustment (metaregression) and bias adjustment.(27,30,31) This suggests that the risk of inconsistency is greatly reduced i”
The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews (2013, background): “Cut-offs based on study size, although scientifically cost-efficient, introduce an extra element of subjectivity and might not ameliorate bias if the remaining large studies are insufficiently critiqued [11].”
A re‐evaluation of fixed effect(s) meta‐analysis (2018, background): “Another example of reweighting based on elicited prior opinion regarding the relevance of each study to a particular scenario was given by Turner et al. (2009).”
GetReal in network meta‐analysis: a review of the methodology (2016, background): “This might be the case, for example, when the included studies refer to populations different than the one targeted by the meta-analysis or when studies without ‘statistically significant’ results have not been published (Turner et al., 2009).”
Checking consistency in mixed treatment comparison meta‐analysis (2010, methodology): “We recommend the exhaustive consideration of potential sources of bias and inconsistency, and suitable bias adjustment prior to synthesis [38].”
Using natural experiments to evaluate population health interventions: new Medical Research Council guidance (2012, methodology): “33 34 If a study has used multiple methods, variation in the estimates should be highlighted.”
A Review of Generalizability and Transportability (2021, methodology): “Bias-adjusted meta-analysis approaches by Turner et al. (2009) and Greenland (2005) require summary-level study outcome data with estimates of bias for each study.”