Recommendations for Increasing Replicability in Psychology †

By Jens B. Asendorpf, Mark Conner, Filip De Fruyt, J. de Houwer, Jaap J. A. Denissen, Klaus Fiedler, Susann Fiedler, D. Funder, Reinhold Kliegl, Brian A. Nosek and 6 more. 2013

769 citations recorded by the source corpus.

Replicability of findings is at the heart of any empirical science. The aim of this article is to move the current replicability debate in psychology towards concrete recommendations for improvement. We focus on research practices but also offer guidelines for reviewers, editors, journal management, teachers, granting institutions, and university promotion committees, highlighting some of the emerging and existing practical solutions that can facilitate implementation of these recommendations. The challenges for improving replicability in psychological science are systemic. Improvement can occur only if changes are made at many levels of practice, evaluation, and reward. Copyright © 2013 John Wiley & Sons, Ltd.

Subfields: Experimental and Cognitive Psychology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/d869ed02a8f1491e126b8c84664af1fc3951751a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Statistics and Probability

Citations in the corpus

The subject-relative advantage in Chinese: Evidence for expectation-based processing (2015, background): “Today, a growing number of methodologically and statistically concerned researchers emphasize the need for replication in experimental psychology (e.g., Asendorpf et al., 2013; Simmons, Nelson, & Simonsohn, 2011).”
Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. (2014, background): “…salient in light of the growing awareness in psychology and other experimental sciences of the problems created and maintained by a proliferation of chronically underpowered studies (Asendorpf et al., 2013; Bakker, van Dijk, & Wicherts, 2012; Button et al., 2013; Ioannidis, 2008; Schimmack, 2012).”
Reproducibility in forecasting research (2015, positive): “Reproduction Re-Analysis Similar distinctions between reproducibility and replicability have been drawn in other scientific disciplines (e.g. in psychology by Asendorpf et al., 2013).”
Recommendations for Replication Research in Special Education (2016, methodology): “…effect size of the replication study falls within the confidence interval of the original study, (b) conducting homogeneity analyses to examine whether study effects are comparable, and (c) meta-analytic aggregation of effects (Asendorpf et al., 2013; Brandt et al., 2014; Valentine et al., 2011).”
The Use of Google Trends in Health Care Research: A Systematic Review (2014, background): “[17,18] Yet, in our study, only 7% of articles provided clear documentation of the necessary fields to be reproducible.”
Best Practice Recommendations for Designing and Implementing Experimental Vignette Methodology Studies (2014, background): “In the case of text or picture vignettes studies, they should be included in the manuscript as this enhances the transparency of the research process (Asendorpf et al., 2013).”
Common Ideological Roots of Speciesism and Generalized Ethnic Prejudice: The Social Dominance Human–Animal Relations Model (SD–HARM) (2016, methodology): “…et al. (2014), in line with the increased awareness of the importance of replicability in psychology to ensure confidence in the findings (e.g. Asendorpf et al., 2013; Nosek, Spies, & Motyl, 2012); (ii) accounting for the possible variance explained by other ideological variables (e.g. RWA,…”
Using independent covariates in experimental designs: Quantifying the trade-off between power boost and Type I error inflation (2017, background): “…find covariates both touted for their power-boosting capabilities and frowned upon for their error-inflating potential within the same paper (e.g., Asendorpf et al., 2013; Ledgerwood, Soderberg, & Sparks, 2017).1 One easy way to reconcile these contrasting narratives is simply to advocate…”
A researcher's guide to regression, discretization, and median splits of continuous variables (2015, background): “…it is assessed after the manipulation but is unaffected by it; this assumption does not seem unreasonable as the field is moving in the direction of larger sample sizes (Asendorpf et al., 2013; Brandt et al., 2014; Cumming, 2014; McShane & Böckenholt, 2014, in press; Pashler & Wagenmakers, 2012).”
The Chicago face database: A free stimulus set of faces and norming data (2015, background): “These advantages seem all the more valuable in light of recent calls for replication and better methodological practices within the field (Asendorpf et al., 2013; Kahneman, 2012; Yong, 2012).”
Psychological measurement and the replication crisis: Four sacred cows. (2020, background): “This crisis encourages us to pause, take, a deep breath, and reconsider our standard means of doing business in psychological science (Asendorpf et al., 2013; Lilienfeld & Waldman, 2017).”