Increasing Transparency Through a Multiverse Analysis

By S. Steegen, Francis Tuerlinckx, Andrew Gelman, Wolf Vanpaemel. Perspectives on Psychological Science, 2016

1,096 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/926763d7946bdf19b0422f35df0b1e4145b324d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

Citations in the corpus

Bayesian Workflow (2020, methodology): “Instead of selecting just one model, we can perform a multiverse analysis and use all of the models and see how the conclusions change across the models (Steegen et al., 2016, Dragicevic et al., 2019, Kale, Kay, and Hullman, 2019).”
Best practice guidance for linear mixed-effects models in psychological science (2019, background): “Steegen et al. (2016) described multiverse analyses, in which all possible data sets are constructed from a sampling of the alternative ways in which raw data can be”
Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It (2017, methodology): “We are certainly not alone in this broad concern: scholars in psychology, for example, have recently mooted the idea of running a given regression analysis on every possible data set that emerges from coding variables differently, and then comparing the resulting p-values (Steegen et al., 2016).(12)”
Three principles of data science: predictability, computability, and stability (PCS) (2018, background): “Others have advocated evaluating re-sults across alternatively processed datasets under the name “multiverse analysis” (22).”
Veridical data science (2019, background): “Others have advocated evaluating re-sults across alternatively processed datasets under the name “multiverse analysis” (23).”
Thinking Clearly About Correlations and Causation: Graphical Causal Models for Observational Data (2017, background): “Jackson, Thoemmes, Jonkmann, Lüdtke, and Trautwein (2012) were interested in the effects of military training (in comparison with civilian community service) on personality.”
Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking (2016, methodology): “…in previous publications (Kriegeskorte et al., 2009; Nieuwenhuis et al., 2011; Simmons et al., 2011; Wagenmakers et al., 2011; Bakker et al., 2012; John et al., 2012; Chambers, 2013; Francis, 2013; Bakker and Wicherts, 2014; Simonsohn et al., 2014; Steegen et al., 2016; van Aert et al., 2016).”
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results (2018, methodology): “For a lone analyst working without the benefit of a crowd, we recommend use of a specification curve (Simonsohn, Simmons, & Nelson, 2016) or multiverse analysis (Steegen et al., 2016).”