Scalable and accurate deep learning with electronic health records
By Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Michaela Hardt, Peter J. Liu, Xiaobing Liu, Jake Marcus, Mimi Sun and 25 more. 2018
2,800 citations recorded by the source corpus.
Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality. Constructing predictive statistical models typically requires extraction of curated predictor variables from normalized EHR data, a labor-intensive process that discards the vast majority of information in each patient’s record. We propose a representation of patients’ entire raw EHR records based on the Fast Healthcare Interoperability Resources (FHIR) format. We demonstrate that deep learning methods using this representation are capable of accurately predicting multiple medical events from multiple centers without site-specific data harmonization. We validated our approach using de-identified EHR data from two US academic medical centers with 216,221 adult patients hospitalized for at least 24 h. In the sequential format we propose, this volume of EHR data unrolled into a total of 46,864,534,945 data points, including clinical notes. Deep learning models achieved high accuracy for tasks such as predicting: in-hospital mortality (area under the receiver operator curve [AUROC] across sites 0.93–0.94), 30-day unplanned readmission (AUROC 0.75–0.76), prolonged length of stay (AUROC 0.85–0.86), and all of a patient’s final discharge diagnoses (frequency-weighted AUROC 0.90). These models outperformed traditional, clinically-used predictive models in all cases. We believe that this approach can be used to create accurate and scalable predictions for a variety of clinical scenarios. In a case study of a particular prediction, we demonstrate that neural networks can be used to identify relevant information from the patient’s chart.Artificial intelligence: Algorithm predicts clinical outcomes for hospital inpatientsArtificial intelligence outperforms traditional statistical models at predicting a range of clinical outcomes from a patient’s entire raw electronic health record (EHR). A team led by Alvin Rajkomar and Eyal Oren from Google in Mountain View, California, USA, developed a data processing pipeline for transforming EHR files into a standardized format. They then applied deep learning models to data from 216,221 adult patients hospitalized for at least 24 h each at two academic medical centers, and showed that their algorithm could accurately predict risk of mortality, hospital readmission, prolonged hospital stay and discharge diagnosis. In all cases, the method proved more accurate than previously published models. The authors provide a case study to serve as a proof-of-concept of how such an algorithm could be used in routine clinical practice in the future.
Subfields: Artificial Intelligence, Health Informatics, Health Information Management, Medical & Health AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Quoc V. Le
- Percy Liang
- Sergey Levine
- Ruslan Salakhutdinov
- Jianfeng Gao
- Kyunghyun Cho
In Health Informatics
- Marzyeh Ghassemi
- Matthew B. A. McDermott
- M. Sadegh Riazi
- T. Marwick
- Zeeshan Ahmed
- Arvid Lundervold
- Ian J. Goodfellow
- Fei Wang
In Health Information Management
- Fei Wang
- Chengxi Zang
- Anna Goldenberg
- Edward Choi
- Slobodan Vucetic
- Tian Bai
- Shanshan Zhang
- Brian L. Egleston
In Medical & Health AI
- Edward Choi
- Fei Wang
- Katherine A. Heller
- Anna Goldenberg
- David Kristjanson Duvenaud
- Yulia Rubanova
- Ricky T. Q. Chen
- B. Van calster
Citations in the corpus
- Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values (2018, methodology): “As DNNs get deployed in areas like medical diagnosis (Rajkomar et al., 2018) and imaging (Lakhani & Sundaram, 2017), reliance on explanations has grown.”
- An overview of deep learning in medical imaging focusing on MRI (2018, background): “seizures [17] and cardiac arrests [18], to computer-aided detection [19] and diagnosis [20] supporting clinical decision making and survival analysis [21], to drug discovery [22] and as an aid in therapy selection and pharmacogenomics [23], to increased operational efficiency [24], stratified care d”
- AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, background): “Although numerous machine learning models, such as neural networks [41,42] and ensemble learning models [43,44], have been developed to complement traditional regression models, most of them are black boxes that do not explain their predictions in a way that humans can understand.”
- Interpretable Representation Learning for Healthcare via Capturing Disease Progression through Time (2018, background): “Recently, [24] developed an ensemble model for several healthcare prediction tasks.”
- Analyzing the role of model uncertainty for electronic health records (2019, methodology): “[26], we train deep RNNs for our clinical tasks.”
- Information Leakage in Embedding Models (2020, background): “’20, November 9–13, 2020, Virtual Event, USA © 2020 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-7089-9/20/11. https://doi.org/10.1145/3372297.3417270 the medical sciences for diagnosis [8, 56]. These advances rely on not only improved training algorithms and architectures, but also ac”
- Deep Reinforcement Learning (2023, neutral): “Numerous issues have been tackled with XRL, including healthcare [34], finance [35], and robotics [36].”
- The need for uncertainty quantification in machine-assisted medical decision making (2019, methodology): “…DL methods have been used in the predictive scenarios related to quality of care, and clinical outcomes where large neural networks were used as function estimators in place of classical predictive models, with reported performance better than the state-of-the-art, classical model approaches 9–11 .”
- Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks (2019, methodology): “, 2017) all use time-agnostic evaluation protocols, which do not account for a significant source of error that would affect models during true deployment: namely, the evolution of care practices over time and the resultant concept drift (Žliobaitė, 2010), which are known to induce significant diffe”
- Key challenges for delivering clinical impact with artificial intelligence (2019, background): “…scores for transfer to intensive care [33], predicting in-hospital mortality, readmission risk, prolonged length of stay and discharge diagnoses [34], predicting future deterioration, including acute kidney injury [35], improving decision-making strategies, including weaning of mechanical…”
- SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records (2021, background): “) based on longitudinal electronic health record (EHR) data [1], [2], [6] show great potentials in improving health care.”
- ARTIFICIAL INTELLIGENCE AND PERSONALIZED MEDICINE (2019, background): “[ 52–54] In addition to the creation of learning systems, there are many initiatives to aggregate data on patients and patient materials to enable data mining and AI-based analyses, for example in cancer contexts,[55] but also for more general settings as well.”
- Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction (2020, methodology): “For performance measurement, we used AUC as our primary evaluation metric, which has been widely adopted by many previous studies of disease prediction [12,14,49].”