The statistical crisis in science

By Andrew Gelman, E. Loken. 2014

808 citations recorded by the source corpus.

Subfields: History and Philosophy of Science, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In History and Philosophy of Science

In Statistics and Probability

Citations in the corpus

Algorithmic stability for adaptive data analysis (2015, background): “prevalence of false discovery arises from the inherent adaptivity in the data analysis process—the fact that the choice of analyses to perform depends on previous interactions with the data (see e.g. [GL14]). Adaptivity is essentially unavoidable when a sequence of research groups publish research p”
The Ladder: A Reliable Leaderboard for Machine Learning Competitions (2015, background): “Finally, in the context of false discovery in the empirical sciences (Ioannidis, 2005; Gelman & Loken, 2014), one could imagine using the the Ladder mechanism as a way of keeping track of scientific progress on important public data sets.”
Private selection from private candidates (2018, background): “This problem, sometimes referred to as the garden of forking paths [16], can lead to a breakdown of standard statistical guarantees.”
The ASA Statement on p-Values: Context, Process, and Purpose (2016, background): “For example, there was considerable discussion about how best to address the issue of multiple potential comparisons (Gelman and Loken 2014).”
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results (2018, background): “Despite methodologists’ remonstrations (Bakker, van Dijk, & Wicherts, 2012; Gelman & Loken, 2014; Simmons, Nelson, & Simonsohn, 2011), it is easy to overlook the fact that results may depend on the chosen analytic strategy, which itself is imbued with theory, assumptions, and choice points.”
For and Against Methodologies: Some Perspectives on Recent Causal and Statistical Inference Debates (2017, background): “…‘‘replication crisis’’, one explanation for null bias is fear of the widespread bias away from the null produced by searching for ‘‘significant’’ results to report (fishing, significance questing, P-hacking) or by data-driven model selection (Leamer 1978; Ioannidis 2008; Gelman and Loken 2014a).”
Visualization in Bayesian workflow (2017, methodology): “A diﬀerent objection to using the data twice (or even more) comes from ideas around hypothesis testing and unbiased estimation, but we are of the opinion that the danger of overﬁtting the data is much more concerning (Gelman and Loken, 2014).”
Increasing Transparency Through a Multiverse Analysis (2016, background): “It is closely related to the idea of a garden of forking paths in data analysis (Gelman & Loken, 2014), which highlights that the one-to-many mapping from scientific theories to statistical hypotheses typically leads to an implicit, potential multiple comparison problem.”