Power failure: why small sample size undermines the reliability of neuroscience

By K. Button, John P. A. Ioannidis, C. Mokrysz, Brian A. Nosek, J. Flint, E. Robinson, Marcus Munafò. Nature Reviews Neuroscience, 2013

7,209 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Computational Theory and Mathematics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c2bc880156c10fc8d1752cd3d528e4ef96cbaac4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

Big data, bigger dilemmas: A critical review (2015, background): “In such samples, a change in a single unit can reverse the significance and affect the reproducibility of results (Button et al., 2013).”
An open investigation of the reproducibility of cancer biology research (2014, neutral): “observed the median power of studies to be 21% (Button et al., 2013), which means that assuming the finding being investigated is true and accurately estimated, then only 21 of every 100 studies investigating that effect would detect statistically significant evidence for the effect.”
Psychological measurement and the replication crisis: Four sacred cows. (2020, background): “These caveats notwithstanding, the statistical power of investigations in human neuroscience remains low on average and is lower than in most other domains of psychology (Button et al., 2013; Carter, Tilling, & Munafò, 2017; Grabitz et al., 2018; Turner, Paul, Miller, & Barbey, 2018).”
A Gentle Introduction to Bayesian Analysis: Applications to Developmental Research (2013, neutral): “When the sample size is small, it is often hard to attain statistical signi ﬁ cant or meaningful results (e.g., Button, et al., 2013).”
Birds of a Caste - How Caste Hierarchies Manifest in Retweet Behavior of Indian Politicians (2020, background): “On the other hand, their low strength in Indian legislature makes it technically difficult to consider the separate castes among them since it could produce an effect of lowering the power of statistical inference among other statistical issues [19].”
How (not) to measure infant Theory of Mind: Testing the replicability and validity of four non-verbal measures (2018, background): “…experimental findings have recently taken center-stage in methodological debates about the evidential status of psychological research (Bakker, van Dijk, & Wicherts, 2012; Button et al., 2013; Makel, Plucker, & Hegarty, 2012; Simmons, Nelson, & Simonsohn, 2011; Simonsohn, Nelson, & Simmons, 2014).”
Is psychology suffering from a replication crisis? What does "failure to replicate" really mean? (2015, background): “Similarly, we agree that the continuation of underpowered studies in many areas of psychology (e.g., Button et al., 2013) undermines scientific psychology.”
The statistical significance filter leads to overoptimistic expectations of replicability (2018, methodology): “butler with the cup/tie accidently shattered today in the dining room. These are implausible because letters can’t shatter. Here, theory predicts a facilitation effect at shattered due to misretrieval of the non-subject cup (vs. tie) (details are discussed in Engelmann et al., 2018). In the number a”
The natural selection of bad science (2016, background): “Less clear, perhaps, is that low power can also increase the false discovery rate and the likelihood that reported effect sizes are inflated, due to their reduced ability to mute stochastic noise [13,63,64].”
The Oxytocin Receptor: From Intracellular Signaling to Behavior. (2018, background): “, invalid statistical means and biased publication, also exist in other fields of psychology (48, 937) or neuroscience in general (141), the increasing number of studies on the overwhelming effects of i.”
Quality Uncertainty Erodes Trust in Science (2017, background): “…asymmetry between authors and readers of scientific products has led to quality uncertainty, and driven the average quality of scientific papers down, to the point where there is now widespread doubt about the robustness of most of the scientific literature (Button et al., 2013; Ioannidis, 2005).”