Data from Paper “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”

By Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn. 2014

1,446 citations recorded by the source corpus.

The data includes measures collected for the two experiments reported in “False-Positive Psychology” [1] where listening to a randomly assigned song made people feel younger (Study 1) or actually be younger (Study 2). These data are useful because they illustrate inflations of false positive rates due to flexibility in data collection, analysis, and reporting of results. Data are useful for educational purposes.

Subfields: Applied Psychology, Cognitive Neuroscience, General Decision Sciences

View Paper: https://www.semanticscholar.org/paper/8a523973e08d08fb6b1c01971b0ab881d09553b3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

In Cognitive Neuroscience

In General Decision Sciences

Citations in the corpus

Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research (2013, background): “Psychologists are under increasing criticism for undisclosed flexibility in data collection and statistical analysis [48].”
Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future (2017, background): “Simmons et al. (2011) illustrated the danger of such procedures by showing that one could literally find evidence for any claim, no matter how absurd, when using these questionable analytical practices, such as the finding that listening to certain types of songs could change people’s actual age.”
Recommendations for Increasing Replicability in Psychology † (2013, background): “…that statistical tools are not robust to the following: (1) optional stopping (adding more cases depending on the outcome of preliminary analyses); (2) data fishing; (3) deletion of cases or outliers for arbitrary reasons; and (4) other common ‘tricks’ to reach significance (Simmons et al., 2011).”
Answers to 18 Questions About Open Science Practices (2018, positive): “A second benefit to pre-registration lies in transparently disclosing and drawing a clear line between the confirmatory versus exploratory aspects of research (Kepes & McDaniel, 2013; Simmons et al., 2011).”
How to make causal inferences using texts (2018, neutral): “Having described our strategy, we connect our approach to existing prior work before demonstrating how it works in two different applications.”
Evidence-based policymaking: promise, challenges and opportunities for accounting and financial markets research** (2018, neutral): “With discretion, researchers might engage in ‘p-hacking’ or ‘fishing’ by estimating many (reasonable) specifications and selecting those that deliver significant results (e.g. Simmons et al. 2011).”
Reducing Implicit Prejudice (2013, background): “Further, there are also many reasons for a study to falsely go " right " – i.e., show a difference where there is not one (Greenwald, 1975; Simmons, Nelson, & Simonsohn, 2011).”
How Much Does Your Data Exploration Overfit? Controlling Bias via Information Usage (2015, background): “samples from N (0, 1) and φI ∗ ∼ N (μ, 1) for μ ∈ [1, 4].”
Data-driven hypothesis weighting increases detection power in genome-scale multiple testing (2016, neutral): “Thus, allowing the researcher to set this parameter can lead to a problem similar to p–value hacking or researcher degrees of freedom [37]: even though the statistical produre, as reported in the published manuscript (with a fixed threshold reported) is valid, the actual validity is contingent on wh”