Small sample sizes reduce the replicability of task-based fMRI studies
By Benjamin O. Turner, E. Paul, Michael B. Miller, Aron K. Barbey. Communications Biology, 2018
345 citations recorded by the source corpus.
Despite a growing body of research suggesting that task-based functional magnetic resonance imaging (fMRI) studies often suffer from a lack of statistical power due to too-small samples, the proliferation of such underpowered studies continues unabated. Using large independent samples across eleven tasks, we demonstrate the impact of sample size on replicability, assessed at different levels of analysis relevant to fMRI researchers. We find that the degree of replicability for typical sample sizes is modest and that sample sizes much larger than typical (e.g., N = 100) produce results that fall well short of perfectly replicable. Thus, our results join the existing line of work advocating for larger sample sizes. Moreover, because we test sample sizes over a fairly large range and use intuitive metrics of replicability, our hope is that our results are more understandable and convincing to researchers who may have found previous results advocating for larger samples inaccessible. Benjamin Turner et al. report an assessment of replicability of task-based fMRI studies as a function of sample size. They find that the degree of replicability for typical samples sizes reported in the literature is modest and advocate for standards requiring larger sample sizes, potentially in excess of N = 100.
Subfields: Cognitive Neuroscience, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/93f72994b7680089813e212efe07e54242a2e6a7
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
In Radiology, Nuclear Medicine and Imaging
Citations in the corpus
- Sample size evolution in neuroimaging research: an evaluation of highly-cited studies (1990-2012) and of latest practices (2017-2018) in high-impact journals (2019, methodology): “Consequently, many false negative outcomes, imprecise measurements, exaggerated published statistically significant effect sizes and high false report probability can be expected in this field (Desmond and Glover, 2002; Murphy and Garavan, 2004; Yarkoni, 2009; Ingre, 2013; Lindquist et al., 2013; Io”
- Evaluation of a decided sample size in machine learning applications (2023, background): “An insufficient sample size has insignificant statistical power, which causes an adverse impact on the true effect and the reproducibility of the findings [22, 23].”