The Replication Recipe: What Makes for a Convincing Replication?

By Mark J. Brandt, H. Ijzerman, A. Dijksterhuis, Frank J. Farach, Jason Geller, Roger Giner-Sorolla, J. Grange, Marco Perugini, Jeffrey R. Spies, A. V. Veer. 2014

604 citations recorded by the source corpus.

Psychological scientists have recently started to reconsider the importance of close replications in building a cumulative knowledge base; however, there is no consensus about what constitutes a convincing close replication study. To facilitate convincing close replication attempts we have developed a Replication Recipe, outlining standard criteria for a convincing close replication. Our Replication Recipe can be used by researchers, teachers, and students to conduct meaningful replication studies and integrate replications into their scholarly habits.

Subfields: Applied Psychology, Experimental and Cognitive Psychology, General Decision Sciences

View Paper: https://www.semanticscholar.org/paper/e19c4efd3587a9a2269a722e4aa15f812d92e5fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In Experimental and Cognitive Psychology

In General Decision Sciences

Citations in the corpus

Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. (2014, background): “…2012; Ioannidis, 2012; Koole & Lakens, 2012; Nosek, Spies, & Motyl, 2012; Open Science Collaboration, 2012) and also emphasized that replication attempts should ideally have high statistical power, even when (perhaps especially when) the original study was underpowered (Brandt et al., 2014).”
Constraints on Generality (COG): A Proposed Addition to All Empirical Papers (2017, background): “…specification of the target population for generalizability and for clearer identification of the parameters critical for observing an effect (e.g., Brandt et al., 2014; Clark, 1973; Greenwald, Pratkanis, Leippe, & Baumgardner, 1986; Henrich et al., 2010; Kukull & Ganguli, 2012; Shoda, Wilson,…”
Estimating the Reproducibility of Experimental Philosophy (2018, methodology): “Once the post-replication form was completed, replication teams were instructed to upload it, along with the all relevant data and documentation, to the corresponding OSF depository, and to register their results using the post-registration form of the Replication Recipe (Brandt et al., 2014) if pos”
Reflections on the replication corner: In praise of conceptual replications (2015, background): “Brandt et al. (2014) have gone so far as to put forth a “replication recipe” for “close” replications. As we explained earlier, we don't believe that “exact” replications are possible. In any given study, researchers make dozens of decisions about small details of procedure, participants, stimuli, s”
Examination of Real-Time Fluctuations in Suicidal Ideation and Its Risk Factors: Results From Two Ecological Momentary Assessment Studies (2017, positive): “This specific replication paradigm (i.e., close replication; Brandt et al., 2014; Finkel, Eastwick, & Reis, 2015) is important because beyond the need to know if our findings replicate in general, it is important to know if our findings generalize across the spectrum of severity of suicide risk.”
Where Have All the “Workers” Gone? A Critical Analysis of the Unrepresentativeness of Our Samples Relative to the Labor Market in the Industrial–Organizational Psychology Literature (2015, background): “It is clear in other fields (e.g., social psychology) that replication is becoming more important (Brandt et al., 2014; Nosek, Spies, & Motyl, 2012; Pashler & Harris, 2012), but published direct replications are extremely rare in psychology as a whole (Makel, Plucker, & Hegarty, 2012).”
Large-Scale Replication Projects in Contemporary Psychological Research (2017, background): “…as the operationalization of the dependent measure(s), the operationalization of the experimental manipulation(s), and unaccounted for moderators but also seemingly minor factors such as the social context, the subject pool, and the time of day (for a comprehensive list, see Brown et al. 2014).”
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education (2018, background): “Alternatively, researchers can complete one of several templates prior to uploading to a registry (e.g., Brandt et al., 2014; van ’t Veer & Giner-Sorolla, 2016; or those maintained by the Center for Open Science, https://osf.io/zab38/wiki/home/, or Project TIER,…”
Investigating Variation in Replicability: A “Many Labs” Replication Project (2022, positive): “Replication is a central tenet of science; its purpose is to confirm the accuracy of empirical findings, clarify the conditions under which an effect can be observed, and estimate the true effect size (Brandt et al., 2013; Open Science Collaboration, 2012, 2014).”