Reproducible brain-wide association studies require thousands of individuals
By Scott A. Marek, B. Tervo-Clemmens, Finnegan J. Calabro, D. Montez, B. Kay, Alexander S. Hatoum, M. Donohue, William Foran, Ryland L. Miller, T. Hendrickson and 35 more. Nature, 2022
1,699 citations recorded by the source corpus.
Magnetic resonance imaging (MRI) has transformed our understanding of the human brain through well-replicated mapping of abilities to specific structures (for example, lesion studies) and functions1–3 (for example, task functional MRI (fMRI)). Mental health research and care have yet to realize similar advances from MRI. A primary challenge has been replicating associations between inter-individual differences in brain structure or function and complex cognitive or mental health phenotypes (brain-wide association studies (BWAS)). Such BWAS have typically relied on sample sizes appropriate for classical brain mapping4 (the median neuroimaging study sample size is about 25), but potentially too small for capturing reproducible brain–behavioural phenotype associations5,6. Here we used three of the largest neuroimaging datasets currently available—with a total sample size of around 50,000 individuals—to quantify BWAS effect sizes and reproducibility as a function of sample size. BWAS associations were smaller than previously thought, resulting in statistically underpowered studies, inflated effect sizes and replication failures at typical sample sizes. As sample sizes grew into the thousands, replication rates began to improve and effect size inflation decreased. More robust BWAS effects were detected for functional MRI (versus structural), cognitive tests (versus mental health questionnaires) and multivariate methods (versus univariate). Smaller than expected brain–phenotype associations and variability across population subsamples can explain widespread BWAS replication failures. In contrast to non-BWAS approaches with larger effects (for example, lesions, interventions and within-person), BWAS reproducibility requires samples with thousands of individuals. Combined data from three large studies, with a total sample size of around 50,000 individuals, indicate that many previous studies linking the brain to complex phenotypes have been statistically underpowered, producing inflated and irreproducible effects.
Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/a0ccfc56e9e22f8d0a073d98d4d1eaf1d8244c0f
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
In Experimental and Cognitive Psychology
Citations in the corpus
- Quantitative assessment of signal quality and usability of EEG and EMG recordings with PEDOT:PSS-coated microneedle electrodes (2025, unknown): “Future work with larger, independent cohorts is necessary to ensure reproducibility and generalizability, in line with recent calls for enhancing reliability in neuroscience research (Marek et al., 2022).”
- Considerations for conducting developmental neuroimaging research on mental health topics (2025, unknown): “Moreover, effect sizes of brain-behavior correlations are relatively low and therefore may require thousands of participants for reproducibility [91].”
- A scoping review of electroencephalographic (EEG) markers for tracking neurophysiological changes and predicting outcomes in substance use disorder treatment (2022, background): “, 2020) brain function studies (Marek et al., 2022) to have adequate statistical power (Grady et al.”