Reproducibility vs. Replicability: A Brief History of a Confused Terminology

By Hans Ekkehard Plesser. Front. Neuroinform., 2018

402 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/dd180a66d2bf55c2f76a5c66cbb6ff0d1d60c817

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Standardisation of quantitative resource conversion studies with black soldier fly larvae (2020, background): “Replicability means that when another team uses the same measurement procedure, the same measuring system, under the same operating conditions, in the same or a different location on multiple trials, they would arrive at the same results (Plesser, 2018).”
Deep learning in clinical natural language processing: a methodical review (2019, background): “A renewed emphasis on scientific rigor has permeated the academic establishment, 108,109 with replicability and reproducibility studies (eg, from ACM’s definitions 110,111 ) showing how much of the literature has fallen short of its scientific goals.”
On the State of Reporting in Crowdsourcing Experiments and a Checklist to Aid Current Practices (2021, result): “Efforts in this regard have been branded under the definitions of repeatability , replicability and reproducibility , to denote attempts at obtaining similar results, by the same or different teams and experimental conditions, under acceptable margins of error [56].”
HARK Side of Deep Learning - From Grad Student Descent to Automated Machine Learning (2019, background): “One essential contribution to this crisis in ML and especially in DL research is the lack of understanding of distinction between repeatability and reproducibility [83].”
Are we really making much progress? A worrying analysis of recent neural recommendation approaches (2019, methodology): “2Precisely speaking, we used a mix of replication and reproduction [12, 35], i.”
A standard protocol for reporting species distribution models (2020, methodology): “…of research reproducibility (Goodman et al. 2016), methods reproducibility means that sufficient details are provided on data and methods in order to independently repeat the study, while results reproducibility means that the same results can be obtained from an independent study (Plesser 2018).”
On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research (2023, neutral): “The exact definition of “repro-ducibility” in computational sciences has been extensively discussed (Claerbout and Karrenbach, 1992; Peng, 2011; Plesser, 2018; Cohen et al., 2018; Tatman et al., 2018; Zhuang et al., 2022).”