Deriving reproducible biomarkers from multi‐site resting‐state data: An Autism‐based example

By Alexandre Abraham, Michael P. Milham, A. di Martino, Cameron R. Craddock, Dimitris Samaras, Bertrand Thirion, Gaël Varoquaux. NeuroImage, 2016

654 citations recorded by the source corpus.

ABSTRACT Resting‐state functional Magnetic Resonance Imaging (R‐fMRI) holds the promise to reveal functional biomarkers of neuropsychiatric disorders. However, extracting such biomarkers is challenging for complex multi‐faceted neuropathologies, such as autism spectrum disorders. Large multi‐site datasets increase sample sizes to compensate for this complexity, at the cost of uncontrolled heterogeneity. This heterogeneity raises new challenges, akin to those face in realistic diagnostic applications. Here, we demonstrate the feasibility of inter‐site classification of neuropsychiatric status, with an application to the Autism Brain Imaging Data Exchange (ABIDE) database, a large (N=871) multi‐site autism dataset. For this purpose, we investigate pipelines that extract the most predictive biomarkers from the data. These R‐fMRI pipelines build participant‐specific connectomes from functionally‐defined brain areas. Connectomes are then compared across participants to learn patterns of connectivity that differentiate typical controls from individuals with autism. We predict this neuropsychiatric status for participants from the same acquisition sites or different, unseen, ones. Good choices of methods for the various steps of the pipeline lead to 67% prediction accuracy on the full ABIDE data, which is significantly better than previously reported results. We perform extensive validation on multiple subsets of the data defined by different inclusion criteria. These enables detailed analysis of the factors contributing to successful connectome‐based prediction. First, prediction accuracy improves as we include more subjects, up to the maximum amount of subjects available. Second, the definition of functional brain areas is of paramount importance for biomarker discovery: brain areas extracted from large R‐fMRI datasets outperform reference atlases in the classification tasks. HIGHLIGHTSWe propose a fully‐automatic pipeline to extract biomarkers from resting state fMRI.We demonstrate prediction in a clinical setting, on subjects coming from unseen site.On 871 subjects of the ABIDE dataset we achieve prediction accuracy better than state of the art (68%).A post‐hoc analysis of the pipeline steps sketches an ideal pipeline for prediction.Extracted autism biomarkers are stable across training sets and consistent with literature.

Subfields: Cognitive Neuroscience, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/76b1dbc306d7710a9ac4a09862c7b36d549400f5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Cross‐validation failure: Small sample sizes lead to large error bars (2017, background): “Moreover, Abraham et al. (2017) and Dansereau et al. (2017) show that with a large number of sites, prediction across sites performed as well as prediction across subjects in the same site.”
Disease prediction using graph convolutional networks: Application to Autism Spectrum Disorder and Alzheimer's disease (2018, methodology): “y Abraham et al. (2017) that met the imaging quality and pheotypic information criteria, comprising 403 individuals with ASD nd 468 healthy controls.”
Benchmarking functional connectome‐based predictive models for resting‐state fMRI (2019, positive): “…and mental health (Craddock et al., 2009; Richiardi et al., 2010; Milazzo et al., 2014; Smith et al., 2015; Miller et al., 2016; Colclough et al., 2017; Dubois et al., 2018), aging (Liem et al., 2017) as well as brain pathologies (Drysdale et al., 2016; Abraham et al., 2017; Ng et al., 2017).”
Spectral Graph Convolutions for Population-based Disease Prediction (2017, methodology): “The use of 10-folds facilitates the comparison with the ABIDE state of the art [1] where a similar strategy is adopted.”
BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis (2020, methodology): “In brain analysis, the reproducibility and 579 consistency of the methods are important [48,1].”
Random Forest Algorithm for the Classification of Neuroimaging Data in Alzheimer's Disease: A Systematic Review (2017, background): “As already demonstrated for rs-fMRI datasets from diﬀerent sites (Abraham et al., 2017; Dansereau et al., 2017), the accuracy and the reliability of the biomarkers extraction could be enhanced by dramatically increasing the cohort size.”
ASD-DiagNet: A Hybrid Learning Approach for Detection of Autism Spectrum Disorder Using fMRI Data (2019, methodology): “Machine learning techniques such as Support Vector Machines (SVM) and random forests are explored in multiple studies (Abraham et al., 2017; Subbaraju et al., 2017; Bi et al., 2018b; Fredo et al., 2018).”
Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks (2017, methodology): “In order to demonstrate the learned metric’s ability to facilitate a subject classification task (ASD vs control), we use a simple k-nn classifier with k = 3 based the estimated distances, and summarise results in Table 1.”
Statistical harmonization corrects site effects in functional connectivity measurements from multi-site fMRI data (2018, background): “Many multi-site studies have reported considerable site or scanner effects in fMRI data (Abraham et al., 2017; Brown et al., 2011; Dansereau et al., 2017; Feis et al., 2015; Friedman et al., 2006, 2008; Forsyth et al., 2014; Gountouna et al., 2010; Jovicich et al., 2016; McGonigle, 2012; Noble et…”
MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites (2016, background): “[41] with the appropriate choice of the data split scheme.”