Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency

By Mallory C. Kidwell, L. Lazarević, E. Baranski, T. Hardwicke, Sarah Piechowski, Lina-Sophia Falkenberg, Curtis Kennett, A. Slowik, Carina M. Sonnleitner, Chelsey Hess-Holden and 3 more. PLoS Biology, 2016

553 citations recorded by the source corpus.

Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.

Subfields: Information Systems and Management, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c10f0d2d0bb5bc92dc33276fc6c8a8a68cff7eda

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

In Statistics and Probability

Citations in the corpus

A methodological synthesis and meta-analysis of judgment tasks in second language research (2020, background): “This situation may change as more journals encourage their authors towards greater transparency (by, for example, adopting the Centre for Open Science badge scheme (Blohowiak et al., 2016) shown to increase long-term availability of materials and data (Kidwell et al., 2016)).”
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support (2020, unknown): “Some journals now award “ open code badges ” (Kidwell et al., 2016) to highlight publications with accessible code and data.”
How open science helps researchers succeed (2016, background): “Initial evidence from a single adopting journal, Psychological Science, and a sample of comparison journals suggests that this simple incentive increases data sharing rates from less than 3% to more than 38% (Kidwell et al., 2016).”
The preregistration revolution (2018, background): “It is possible that such incentives will become nearly as effective as badges for open data, which were associated with more than a 10-fold increase in data sharing in an initial test (58).”
The Replication Crisis in Psychology: An Overview for Theoretical and Philosophical Psychology (2019, background): “Because researchers experience strong pressure to publish in high-impact outlets and because such outlets generally have a reputation for publishing novel and positive findings, critics suggest that these pressures can lead researchers to engage in QRPs that will make their findings more attractive ”
The ARRIVE guidelines 2.0: updated guidelines for reporting animal research (2020, background): “…introduced a range of initiatives to address the problem, from open access and open practices to enable the scrutiny of all aspects of the research (Kidwell et al. 2016, Else, 2018) through to study preregistration to shift the focus towards robust methods rather than the novelty of the results…”
Quality Uncertainty Erodes Trust in Science (2017, background): “…change is rare, and the most successful academic journals are unlikely to change on their own, for fear of harming their reputations (though see Kidwell et al., 2016 and Piwoward, Day, & Fridsma, 2007, for evidence that transparency does not seem to have a negative impact on traditional…”
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education (2018, background): “Offering openscience badges has been associated with an increase in data sharing (Kidwell et al., 2016) in the absence of any mandate to do so.”
Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking (2016, background): “These transparent practices have many benefits and are currently gaining traction (e.g., Eich, 2014; Kidwell et al., 2016), but are often insufficiently detailed, documented or structured to allow other researchers to reproduce and replicate results (e.”
Data Sharing in Psychology: A Survey on Barriers and Preconditions (2018, background): “In terms of effectiveness, Kidwell et al. (2016) found that after the premier journal Psychological Science had adopted the open data badges, data sharing rates increased dramatically.”
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research* (2020, unknown): “The bioscience community has introduced a range of initiatives to address the problem, from open access and open practices to enable the scrutiny of all aspects of the research [4, 5] through to study preregistration to shift the focus towards robust methods rather than the novelty of the results [6”
How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables (2019, background): “One element that could help nudging researchers towards properly powered studies may be the inclusion of power in the badge system used in some journals to promote good research practices (Chambers, 2013; Lindsay, 2015, 2017; Kidwell et al., 2016; Nosek & Lakens, 2014).”