Epilepsyecosystem.org: crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG
By Levin Kuhlmann, Philippa J. Karoly, D. Freestone, B. Brinkmann, A. Temko, Alexandre Barachant, Feng Li, Gilberto Titericz, Brian Lang, Daniel Lavery and 16 more. 2018
185 citations recorded by the source corpus.
Accurate seizure prediction will transform epilepsy management by offering warnings to patients or triggering interventions. However, state-of-the-art algorithm design relies on accessing adequate long-term data. Crowd-sourcing ecosystems leverage quality data to enable cost-effective, rapid development of predictive algorithms. A crowd-sourcing ecosystem for seizure prediction is presented involving an international competition, a follow-up held-out data evaluation, and an online platform, Epilepsyecosystem.org, for yielding further improvements in prediction performance. Crowd-sourced algorithms were obtained via the 'Melbourne-University AES-MathWorks-NIH Seizure Prediction Challenge' conducted at kaggle.com. Long-term continuous intracranial electroencephalography (iEEG) data (442 days of recordings and 211 lead seizures per patient) from prediction-resistant patients who had the lowest seizure prediction performances from the NeuroVista Seizure Advisory System clinical trial were analysed. Contestants (646 individuals in 478 teams) from around the world developed algorithms to distinguish between 10-min inter-seizure versus pre-seizure data clips. Over 10 000 algorithms were submitted. The top algorithms as determined by using the contest data were evaluated on a much larger held-out dataset. The data and top algorithms are available online for further investigation and development. The top performing contest entry scored 0.81 area under the classification curve. The performance reduced by only 6.7% on held-out data. Many other teams also showed high prediction reproducibility. Pseudo-prospective evaluation demonstrated that many algorithms, when used alone or weighted by circadian information, performed better than the benchmarks, including an average increase in sensitivity of 1.9 times the original clinical trial sensitivity for matched time in warning. These results indicate that clinically-relevant seizure prediction is possible in a wider range of patients than previously thought possible. Moreover, different algorithms performed best for different patients, supporting the use of patient-specific algorithms and long-term monitoring. The crowd-sourcing ecosystem for seizure prediction will enable further worldwide community study of the data to yield greater improvements in prediction performance by way of competition, collaboration and synergism.10.1093/brain/awy210_video1awy210media15817489051001.
Subfields: Cognitive Neuroscience, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
In Signal Processing
Citations in the corpus
- A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, background): “hods on some end-task. How can we gather a large collection of entries into such a large matrix? Apart from competitions and challenges, crowd-sourcing efforts will aid the formation of such database [147], [148]. A clear problem is how multidimensional and gigantic such tabulation will become, not ”
- Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review (2020, methodology): “The use of ROC curves as evaluation matric is evident from [95], [114] Area under the ROC curve (AUC) is also used for performance evaluation of ES predicting algorithms [53].”
- Deep Learning for Medical Anomaly Detection – A Survey (2020, background): “Their system is evaluated on 2016 Kaggle seizure prediction competition dataset [97] and the proposed system achieves a 87.”