Single Subject Prediction of Brain Disorders in Neuroimaging: Promises and Pitfalls

By Mohammad R. Arbabshirani, Sergey M. Plis, Jing Sui, Vince D. Calhoun. NeuroImage, 2016

852 citations recorded by the source corpus.

ABSTRACT Neuroimaging‐based single subject prediction of brain disorders has gained increasing attention in recent years. Using a variety of neuroimaging modalities such as structural, functional and diffusion MRI, along with machine learning techniques, hundreds of studies have been carried out for accurate classification of patients with heterogeneous mental and neurodegenerative disorders such as schizophrenia and Alzheimer's disease. More than 500 studies have been published during the past quarter century on single subject prediction focused on a multiple brain disorders. In the first part of this study, we provide a survey of more than 200 reports in this field with a focus on schizophrenia, mild cognitive impairment (MCI), Alzheimer's disease (AD), depressive disorders, autism spectrum disease (ASD) and attention‐deficit hyperactivity disorder (ADHD). Detailed information about those studies such as sample size, type and number of extracted features and reported accuracy are summarized and discussed. To our knowledge, this is by far the most comprehensive review of neuroimaging‐based single subject prediction of brain disorders. In the second part, we present our opinion on major pitfalls of those studies from a machine learning point of view. Common biases are discussed and suggestions are provided. Moreover, emerging trends such as decentralized data sharing, multimodal brain imaging, differential diagnosis, disease subtype classification and deep learning are also discussed. Based on this survey, there is extensive evidence showing the great potential of neuroimaging data for single subject prediction of various disorders. However, the main bottleneck of this exciting field is still the limited sample size, which could be potentially addressed by modern data sharing models such as the ones discussed in this paper. Emerging big data technologies and advanced data‐intensive machine learning methodologies such as deep learning have coincided with an increasing need for accurate, robust and generalizable single subject prediction of brain disorders during an exciting time. In this report, we survey the past and offer some opinions regarding the road ahead. HighlightsPast efforts on classification of brain disorders are comprehensively reviewed.The common pitfalls from machine learning point of view are discussed.Emerging trends related to single‐subject prediction are reviewed and discussed.

Subfields: Cognitive Neuroscience, Neurology

View Paper: https://www.semanticscholar.org/paper/65d009f6c4abd501e452ed6664539ce5f1a733d3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Neurology

Citations in the corpus

Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration (2018, background): “22 Despite all these efforts, only CAD for breast imaging has been widely adopted in clinical practice.”
Artificial Intelligence and the Future of Psychiatry (2020, background): “More than 200 studies have used machine learning algorithms to distinguish brain differences between people with and without particular disorders such as Alzheimer’s disease, schizophrenia, and depression [2].”
Your evidence? Machine learning algorithms for medical diagnosis and prediction (2019, background): “Moreover, the possibility of making individual prognoses in psychiatry using neuroimaging and machine learning is under investigation (Arbabshirani et al., 2017; Bzdok & Meyer-Lindenberg, 2018; Janssen, Mourao-Miranda, & Schnack, 2018).”
Cross‐validation failure: Small sample sizes lead to large error bars (2017, background): “5 15 15 t o 50 50 t o 150 150 t o 500 500 t o 1500 1500 t o 5000 2017) review paper, diﬀerentiating Schizophrenia and Alzheimer’s studies, the Brown and Hamarneh (2016) review on prediction from connec-tomes, and the Wolfers et al. (2015) review on prediction for psychiatric disorders, as well as…”
Multi-Site Diagnostic Classification of Schizophrenia Using Discriminant Deep Learning with Functional Connectivity MRI (2018, background): “(Arbabshirani et al., 2017) for a review).”
Classification and Prediction of Brain Disorders Using Functional Connectivity: Promising but Challenging (2018, background): “One previous study in our group has shown that features can have different distributions but comparable group means for different cohorts (Arbabshirani et al., 2017).”
Reconstructing subject‐specific effect maps (2017, methodology): “nt, maps indicate statistical relationship between the condition and measurement across the entire population. Either a group analysis technique [2,22,40] or a machine learning based predictive model [1,14,26,31,13] is used to compare two cohorts of subjects, one showing the condition of interest an”
Hierarchical Fully Convolutional Network for Joint Atrophy Localization and Alzheimer’s Disease Diagnosis using Structural MRI (2018, background): “…(sMRI) can non-invasively capture profound brain changes induced by the atrophic process [3], based on which various computer-aided diagnosis (CAD) approaches [4], [5], [6] have been proposed for the early diagnosis of AD as well as its pro-dromal stage, i.e., mild cognitive impairment (MCI).”
The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features (2018, neutral): “classifying brain disease patients from controls, which is likely due to the overfitting issue related to the small sample size (Arbabshirani et al., 2017).”
Benchmarking functional connectome‐based predictive models for resting‐state fMRI (2019, positive): “We ﬁ rst survey methods used for prediction studies based on three extensive reviews: Wolfers et al. (2015); Arbabshirani et al. (2017); Brown and Hamarneh (2016).”
Machine learning algorithm validation with a limited sample size (2019, methodology): “[3] conducted a survey of neuroimaging studies which used supervised ML methods to classify healthy individuals and individuals with brain disorders.”
Disease prediction using graph convolutional networks: Application to Autism Spectrum Disorder and Alzheimer's disease (2018, background): “ccuracy, yielding state of the art results Arbabshirani et al. (2017) , braham et al. (2017) and Heinsfeld et al. (2018) for both BIDE (70.”
Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications (2017, neutral): “…four participants) contributed very little for the total sample size (80), it is still possible to misclassify all its cases and achieve a low error rate. such cases, a high accuracy can be misleading, as it may reﬂect an overestimation of the algorithm’s performance (Arbabshirani et al., 2016).”
A review on neuroimaging‐based classification studies and associated feature extraction methods for Alzheimer's disease and its prodromal stages (2017, neutral): “Recent review papers (Arbabshirani et al., 2017; Falahati et al., 2014) reported studies on MRI- and multimodality-based classification of AD and MCI, limiting AD classification to MRI or its combination with other modalities only.”
Machine Learning for Precision Psychiatry: Opportunities and Challenges. (2017, methodology): “As an important practical consequence, machine learning and classical statistics do not judge data on the same aspects of evidence: An observed effect assessed to be statistically significant by a p-value does not in all cases yield a high prediction accuracy in new, independent data, and vice versa”