A manifesto for reproducible science

By Marcus Munafò, Brian A. Nosek, D. Bishop, K. Button, Christopher D. Chambers, N. P. Sert, Uri Simonsohn, Eric-Jan Wagenmakers, J. Ware, John P. A. Ioannidis. Nature Human Behaviour, 2017

2,738 citations recorded by the source corpus.

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.

Subfields: Information Systems and Management, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/a68ce412e92d87ca0116519651bbc484d98c76ae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

In Statistics and Probability

Citations in the corpus

An Agenda for Open Science in Communication (2020, background): “Open science practices can be an important part of ensuring our research is reproducible, replicable, and thus credible (Munafò et al., 2017).”
Estimating the Reproducibility of Experimental Philosophy (2018, background): “Meanwhile, promising ideas for improvement—including the recent push toward norms of pre-registration— are now gaining traction among leading scientists (Chambers & Munafò, 2013; Munafò et al., 2017; Nosek et al., 2018; but see Lash & Vandenbroucke, 2012; Scott, 2013).”
A standard protocol for reporting species distribution models (2020, background): “In summary, ODMAP will help answer the clarion calls for reproducible computational science (Peng 2011), and for improved recording and reporting of methods and data (Mesirov 2010, Munafò et al. 2017), within the field of species distribution modelling, a crucial tool for science and conservation.”
Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) (2020, background): “The implementation of Open science practices has been identified as a core factor that could improve the reproducibility of science (Munafò et al., 2017).”
Reproducibility: Designing a rigorous microscopy experiment: Validating methods and avoiding bias (2019, neutral): “A microscope generates a magnified optical image of the spatial distribution of the fluorophores in the specimen (Inoué and Spring, 1997; Murphy and Davidson, 2012).”
Editorial: Enhancing quantitative theory-testing entrepreneurship research (2019, background): “…with methodological approaches being as varied as they are, we had to start somewhere and chose to focus this editorial on mainstream, quantitative theory-testing research that tests a pre-defined hypothesis using either experimental or observational data (Bergh et al., 2017; Munafò et al., 2017).”
Directions in Abusive Language Training Data: Garbage In, Garbage Out (2020, background): “…sharing would (1) greater collaboration amongst researchers, (2) enhance the reproducibility of research by encouraging greater scrutiny [54, 77, 80] and (3) substantively advance the field by enabling future researchers to better understand the biases and limitations of existing research and…”
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education (2018, background): “Thus, research evidence is likely skewed toward positive findings, with few replication studies conducted to identify potentially false-positive findings (Ioannidis, 2012), leading to a literature base with potentially exaggerated evidence for positive effects (Munafo et al., 2017).”
A methodological synthesis and meta-analysis of judgment tasks in second language research (2020, methodology): “Following Plonsky and Oswald (2015) we included Linguistics and Language Behavior Abstracts (LLBA), Education Resources Information Center (ERIC), PsycInfo, and Google Scholar.”