Dissemination and publication of research findings: an updated review of related biases.

By F. Song, S. Parekh, L. Hooper, Y. Loke, Jonathan J. Ryder, A. Sutton, C. Hing, C. Kwok, C. Pang, I. Harvey. Health Technology Assessment, 2010

995 citations recorded by the source corpus.

OBJECTIVES To identify and appraise empirical studies on publication and related biases published since 1998; to assess methods to deal with publication and related biases; and to examine, in a random sample of published systematic reviews, measures taken to prevent, reduce and detect dissemination bias. DATA SOURCES The main literature search, in August 2008, covered the Cochrane Methodology Register Database, MEDLINE, EMBASE, AMED and CINAHL. In May 2009, PubMed, PsycINFO and OpenSIGLE were also searched. Reference lists of retrieved studies were also examined. REVIEW METHODS In Part I, studies were classified as evidence or method studies and data were extracted according to types of dissemination bias or methods for dealing with it. Evidence from empirical studies was summarised narratively. In Part II, 300 systematic reviews were randomly selected from MEDLINE and the methods used to deal with publication and related biases were assessed. RESULTS Studies with significant or positive results were more likely to be published than those with non-significant or negative results, thereby confirming findings from a previous HTA report. There was convincing evidence that outcome reporting bias exists and has an impact on the pooled summary in systematic reviews. Studies with significant results tended to be published earlier than studies with non-significant results, and empirical evidence suggests that published studies tended to report a greater treatment effect than those from the grey literature. Exclusion of non-English-language studies appeared to result in a high risk of bias in some areas of research such as complementary and alternative medicine. In a few cases, publication and related biases had a potentially detrimental impact on patients or resource use. Publication bias can be prevented before a literature review (e.g. by prospective registration of trials), or detected during a literature review (e.g. by locating unpublished studies, funnel plot and related tests, sensitivity analysis modelling), or its impact can be minimised after a literature review (e.g. by confirmatory large-scale trials, updating the systematic review). The interpretation of funnel plot and related statistical tests, often used to assess publication bias, was often too simplistic and likely misleading. More sophisticated modelling methods have not been widely used. Compared with systematic reviews published in 1996, recent reviews of health-care interventions were more likely to locate and include non-English-language studies and grey literature or unpublished studies, and to test for publication bias. CONCLUSIONS Dissemination of research findings is likely to be a biased process, although the actual impact of such bias depends on specific circumstances. The prospective registration of clinical trials and the endorsement of reporting guidelines may reduce research dissemination bias in clinical research. In systematic reviews, measures can be taken to minimise the impact of dissemination bias by systematically searching for and including relevant studies that are difficult to access. Statistical methods can be useful for sensitivity analyses. Further research is needed to develop methods for qualitatively assessing the risk of publication bias in systematic reviews, and to evaluate the effect of prospective registration of studies, open access policy and improved publication guidelines.

Subfields: Complementary and alternative medicine, General Health Professions, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/894648b75aee4e9b5d6a2ec7b71ebd65e40df9d1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Complementary and alternative medicine

Douglas G. Altman

In General Health Professions

In Statistics and Probability

Citations in the corpus

PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts (2013, background): “This is important as inclusion, or not, of studies published in languages other than English (examples 3c and 3d), unpublished data, or older data can influence the estimates of effect or association in meta-analyses [17,18].”
Scholarship, Research, and Creative Work at Bryn Mawr College Scholarship, Research, and Creative Work at Bryn Mawr College (background): “Todisco himself, in his preface, pays homage to the influential views of Ranuccio Bianchi Bandinelli, here extensively quoted together with other contemporary Italian scholars who stress the unity of the archaeological inventory, whether sherd or statue; he also acknowledges the difficulty of…”
Negative results are disappearing from most disciplines and countries (2011, background): “…or non-US countries have been observed in past meta-analyses, and were usually attributed to editorial and peer-review biases, which might tend to reject papers from certain countries unless they present particularly strong or appealing results (Song et al. 2010; Yousefi-Nooraie et al. 2006).”
Positive Outcomes Influence the Rate and Time to Publication, but Not the Impact Factor of Publications of Clinical Trial Results (2013, background): “Clinical trial publication is not a clearly dichotomic process [1], [18].”
Strengthening statistical usage in marine ecology (2012, background): “Editors should welcome submission of high-quality studies of this type for the fields covered by their journals (e.g., Song et al., 2010).”
The predictive validity of risk assessment tools for female offenders: A systematic review (2015, neutral): “Although such exclusion has been found to have minimal effect in reviews (Juni, Holenstein, Sterne, Bartlett, & Egger, 2002), it may nevertheless introduce systematic bias (Song et al., 2010).”
Publication Bias:A call for improved meta-analytic practice in the organizational sciences (2012, background): “Journal articles, books, and tutorials provide guidance on how to address the issue of publication bias in metaanalytic reviews (e.g., Berlin & Ghersi, 2005; Borenstein, Hedges, Higgins, & Rothstein, 2009; Hunter & Schmidt, 2004; McDaniel et al., 2006; Rothstein, Sutton, & Borenstein, 2005a; Song et”
Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. (2010, background): “Sample size, median (IQR) [range] 84 (46-206) [4-6848]”