Registered Reports A Method to Increase the Credibility of Published Results

By Brian A. Nosek, Daniël Lakens. 2014

621 citations recorded by the source corpus.

Ignoring replications and negative results is bad for science. This special issue presents a novel publishing format – Registered Reports – as a partial solution. Peer review occurs prior to data collection, design and analysis plans are preregistered, and results are reported regardless of outcome. Fourteen Registered Reports of replications of important published results in social psychology are reported with strong confirmatory tests. Further, the articles demonstrate open science practices such as open data, open materials, and disclosure of research process, conflicts of interest, and contributions. The credibility of published science will increase with cultural shifts that accept replications and negative results as viable research outcomes, and when transparency and reproducibility are part of standard research practice.

Subfields: Artificial Intelligence, General Decision Sciences, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6765332ea2813d8b0c744fe20fb02269c23bfec8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Decision Sciences

In Statistics and Probability

Citations in the corpus

Do p Values Lose Their Meaning in Exploratory Analyses? It Depends How You Define the Familywise Error Rate (2017, background): “…out by Nosek, Wagenmakers, and others (Bender & Lange, 2001; Dahl et al., 2008; de Groot, 1956/2014; Forstmeier et al., 2016; Nosek et al., 2017; Nosek & Lakens, 2014; Wagenmakers, 2016) is that although the precise number of tests is known in preregistered confirmatory analyses, it is not…”
Self-regulation in childhood as a predictor of future outcomes: A meta-analytic review. (2020, methodology): “…best method to remove publication bias from the scientific record is for researchers to adopt registered reports as the preferred method of publication (Chambers, 2013; Nosek & Lakens, 2014) and future meta-analyses to establish registered reports as a criterion for inclusion in the meta-analysis.”
The past, present, and future of ego depletion (2019, background): “Broad tests of the replicability of psychology suggest that the case of ego depletion might be less exception and more rule in social psychology, where a majority of papers have failed the replication test (Klein et al., 2018; Nosek & Lakens, 2014; Open Science Collaboration, 2015).”
Estimating the reproducibility of psychological science (2015, neutral): “Direct replication is the attempt to recreate the conditions believed sufficient for obtaining a previously observed finding ( 7 , 8 ) and is the means of establishing reproducibility of a finding with new data.”
Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field. (2018, methodology): “We have identified many factors that must work together to change production of and attitudes towards replications, including increased transparency of materials and data, multi-site collaboration, more consistent self-labelling of replications, fewer and more transparent alterations of features fro”
Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences (2019, background): “These and other sources include explanations of the importance of replication (e.g., Anderson & Maxwell, 2016; Bettis, Helfat, & Shaver, 2016; Pashler & Wagenmakers, 2012; Simons, 2014; Vazire, 2018) and calls for change to the review process (e.g., Nosek & Lakens, 2014).”
Pre-registration in social psychology—A discussion and suggested template (2016, methodology): “We refer to this model as reviewed pre-registration (RPR), which has also been called a “Registered Report” (Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014; Nosek & Lakens, 2014).”
Choosing prediction over explanation in psychology: Lessons from machine learning (2017, background): “This appreciation is most evident in the numerous recent calls for routine preregistration of studies (Chambers & Munafo, 2013; Nosek & Lakens, 2014), where a sharp distinction is to be drawn between the exploratory (where flexibility is encouraged) and confirmatory (where flexibility is denied)…”
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education (2018, result): “…to increasing the robustness of the observed research results by providing feedback on the preregistered study plans, Registered Reports address possible prejudice against null results by making initial decisions for publication prior to knowing the results of the research (Nosek & Lakens, 2014).”
Many Labs 2: Investigating Variation in Replicability Across Samples and Settings (2018, methodology): “Following the Registered Report model (Nosek & Lakens, 2014), prior to data collection we submitted the materials and protocols to formal peer review in a process conducted by this journal’s Editor.”