Publication bias in clinical trials due to statistical significance or direction of trial results.

By Sally Hopewell, Kirsty Loudon, Mike Clarke, Andrew D. Oxman, K. Dickersin. Cochrane Database of Systematic Reviews, 2009

875 citations recorded by the source corpus.

BACKGROUND The tendency for authors to submit, and of journals to accept, manuscripts for publication based on the direction or strength of the study findings has been termed publication bias. OBJECTIVES To assess the extent to which publication of a cohort of clinical trials is influenced by the statistical significance, perceived importance, or direction of their results. SEARCH STRATEGY We searched the Cochrane Methodology Register (The Cochrane Library [Online] Issue 2, 2007), MEDLINE (1950 to March Week 2 2007), EMBASE (1980 to Week 11 2007) and Ovid MEDLINE In-Process & Other Non-Indexed Citations (March 21 2007). We also searched the Science Citation Index (April 2007), checked reference lists of relevant articles and contacted researchers to identify additional studies. SELECTION CRITERIA Studies containing analyses of the association between publication and the statistical significance or direction of the results (trial findings), for a cohort of registered clinical trials. DATA COLLECTION AND ANALYSIS Two authors independently extracted data. We classified findings as either positive (defined as results classified by the investigators as statistically significant (P < 0.05), or perceived as striking or important, or showing a positive direction of effect) or negative (findings that were not statistically significant (P >/= 0.05), or perceived as unimportant, or showing a negative or null direction in effect). We extracted information on other potential risk factors for failure to publish, when these data were available. MAIN RESULTS Five studies were included. Trials with positive findings were more likely to be published than trials with negative or null findings (odds ratio 3.90; 95% confidence interval 2.68 to 5.68). This corresponds to a risk ratio of 1.78 (95% CI 1.58 to 1.95), assuming that 41% of negative trials are published (the median among the included studies, range = 11% to 85%). In absolute terms, this means that if 41% of negative trials are published, we would expect that 73% of positive trials would be published.Two studies assessed time to publication and showed that trials with positive findings tended to be published after four to five years compared to those with negative findings, which were published after six to eight years. Three studies found no statistically significant association between sample size and publication. One study found no significant association between either funding mechanism, investigator rank, or sex and publication. AUTHORS' CONCLUSIONS Trials with positive findings are published more often, and more quickly, than trials with negative findings.

Subfields: History and Philosophy of Science, Pharmacology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e992a912f79adfbbb16c6d7d3f0e34c7049f5a06

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In History and Philosophy of Science

In Pharmacology

In Statistics and Probability

Citations in the corpus

A Call for Examining Replication and Bias in Special Education Research (2014, result): “Similarly, among 21 studies that used both blinded and non-blinded observations of binary outcomes, Hróbjartsson et al. (2012) found that non-blinded observations resulted in a 36% exaggeration of odds ratios. When analyzing data, researchers may bias results by failing to meet the assumptions of st”
Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. (2010, background): “However,weknowthattrials withpositivefindingshavenearly4times the odds of being published compared with those with negative findings.(15) To theextentthatpublicationbias ispresent, inclusion of unpublished studies would lead to a diminished pooled effect from the nontruncated RCTs.”
Meta‐regression approximations to reduce publication selection bias (2014, background): “Nonetheless, a recent systematic review found that publication selection is quite common in medical research (Hopewell et al. 2009).”
Scholarship, Research, and Creative Work at Bryn Mawr College Scholarship, Research, and Creative Work at Bryn Mawr College (background): “Todisco himself, in his preface, pays homage to the influential views of Ranuccio Bianchi Bandinelli, here extensively quoted together with other contemporary Italian scholars who stress the unity of the archaeological inventory, whether sherd or statue; he also acknowledges the difficulty of…”
The use of Electronic Health Records to Support Population Health: A Systematic Review of the Literature (2018, background): “Publication bias is likely to occur because publishers tend to publish articles with significant relationships, and therefore articles that did not result in significant findings were not able to be selected for this review [72].”
Positive Outcomes Influence the Rate and Time to Publication, but Not the Impact Factor of Publications of Clinical Trial Results (2013, background): “When the publication or not of a particular study is determined by its outcomes or statistical significance, the derived bias is unacceptable, since it results in an exaggeration of the positive effects of a given treatment [1], [2].”