Benchmarking deep learning models on large healthcare datasets
By Sanjay Purushotham, Chuizheng Meng, Zhengping Che, Yan Liu. Journal of Biomedical Informatics, 2018
399 citations recorded by the source corpus.
Deep learning models (aka Deep Neural Networks) have revolutionized many fields including computer vision, natural language processing, speech recognition, and is being increasingly used in clinical healthcare applications. However, few works exist which have benchmarked the performance of the deep learning models with respect to the state-of-the-art machine learning models and prognostic scoring systems on publicly available healthcare datasets. In this paper, we present the benchmarking results for several clinical prediction tasks such as mortality prediction, length of stay prediction, and ICD-9 code group prediction using Deep Learning models, ensemble of machine learning models (Super Learner algorithm), SAPS II and SOFA scores. We used the Medical Information Mart for Intensive Care III (MIMIC-III) (v1.4) publicly available dataset, which includes all patients admitted to an ICU at the Beth Israel Deaconess Medical Center from 2001 to 2012, for the benchmarking tasks. Our results show that deep learning models consistently outperform all the other approaches especially when the 'raw' clinical time series data is used as input features to the models.
Subfields: Artificial Intelligence, Pulmonary and Respiratory Medicine, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/1701a367d06fda7d5c17f0345c06dcd093dac7ef
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Anna Goldenberg
- Adrian Weller
- Marzyeh Ghassemi
- Umang Bhatt
- Aram Galstyan
- Tristan Naumann
- José M. F. Moura
- Hrayr Harutyunyan
In Pulmonary and Respiratory Medicine
- Ahmed Hosny
- Hugo J. W. L. Aerts
- Charles Swanton
- Marzyeh Ghassemi
- Matthew B. Schabath
- Matthew B. A. McDermott
In Radiology, Nuclear Medicine and Imaging
- Hugo J. W. L. Aerts
- Ahmed Hosny
- Aram Galstyan
- Maryellen Lissak Giger
- José M. F. Moura
- Charles Swanton
- Marzyeh Ghassemi
- Matthew B. A. McDermott
Citations in the corpus
- A Survey of Healthcare Internet-of-Things (HIoT): A Clinical Perspective (2019, background): “algorithms and HCOs rely on their individual databases [216], [217].”
- A Survey on Machine Learning Techniques for Cyber Security in the Last Decade (2020, background): “However, DL models are data-hungry models that show excellent performance on larger datasets [207] , [208].”
- Multitask learning and benchmarking with clinical time series data (2017, methodology): “ length-of-stay prediction [42, 44, 45], phenotyping [38, 39, 46] and readmission prediction [47]. Additionally, attention-based RNNs were applied for all our benchmark tasks [48]. In a parallel work [49], another set of benchmark tasks based on MIMIC-III was introduced that includes multiple versio”
- Evaluating and Aggregating Feature-based Model Explanations (2020, methodology): “We extract seventeen real-valued features deemed critical, per [ Purushotham et al. , 2018 ] , for sepsis prediction.”
- Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks (2019, methodology): “Models using raw, non-featurised data representations, as advocated by recent deep learning ICU prediction systems such as Purushotham et al. (2018), are universally unable to generalise well across large dataset shifts as exemplified by the 2008 system switch within MIMIC-III.”
- Artificial intelligence in Cancer Imaging: Clinical Challenges and Applications (2019, background): “206 Furthermore, access to available data sets should be improved to promote intellectual collaboration.”