Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks

By Fenglong Ma, Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, Jing Gao. 2017

623 citations recorded by the source corpus.

Predicting the future health information of patients from the historical Electronic Health Records (EHR) is a core research task in the development of personalized healthcare. Patient EHR data consist of sequences of visits over time, where each visit contains multiple medical codes, including diagnosis, medication, and procedure codes. The most important challenges for this task are to model the temporality and high dimensionality of sequential EHR data and to interpret the prediction results. Existing work solves this problem by employing recurrent neural networks (RNNs) to model EHR data and utilizing simple attention mechanism to interpret the results. However, RNN-based approaches suffer from the problem that the performance of RNNs drops when the length of sequences is large, and the relationships between subsequent visits are ignored by current RNN-based approaches. To address these issues, we propose Dipole, an end-to-end, simple and robust model for predicting patients' future health information. Dipole employs bidirectional recurrent neural networks to remember all the information of both the past visits and the future visits, and it introduces three attention mechanisms to measure the relationships of different visits for the prediction. With the attention mechanisms, Dipole can interpret the prediction results effectively. Dipole also allows us to interpret the learned medical code representations which are confirmed positively by medical experts. Experimental results on two real world EHR datasets show that the proposed Dipole can significantly improve the prediction accuracy compared with the state-of-the-art diagnosis prediction approaches and provide clinically meaningful interpretation.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Interpretable Representation Learning for Healthcare via Capturing Disease Progression through Time (2018, methodology): “Many of those tasks boil down to being able to develop a predictive model that could forecast the future state of a patient given the currently available information about the patient [7, 8, 19].”
Forecasting: theory and practice (2020, methodology): “publishing real-time accurate forecasts of confirmed cases and deaths from mid-March 2020 onwards. Their approach is based on extraction of trends from the data using machine learning. Pinson and Makridakis (2020) organised a debate between Taleb and Ioannidis on forecasting pandemics.”
Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review (2018, methodology): “41,45 The attention mechanism is also used to derive a latent representation of medical codes (eg., diagnosis codes, medication codes).”
Attentive State-Space Modeling of Disease Progression (2019, methodology): “The models in [14, 15, 16] use a reverse-time attention mechanism to learn visit-level attention weights that explain the predictions of an RNN.”
Attention Models in Graphs (2018, methodology): “Finally, Dipole [Ma et al. 2017] applied an attention model similar to that of [Bahdanau et al. 2015] to sequential medical data.”
Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer (2019, background): “…tasks such as predicting diagnosis (Lipton et al. 2015; Choi et al. 2016a; Rajkomar et al. 2018), learning medical concept representations (Che et al. 2015; Choi, Chiu, and Sontag 2016; Choi et al. 2016b; Miotto et al. 2016), and making interpretable predictions (Choi et al. 2016c; Ma et al. 2017).”
Doctor XAI: an ontology-based approach to black-box sequential data classification explanations (2020, background): “Many interpretability approaches related to sequential data modeling focus on adding an attention mechanism [2, 44] to a sequential model and use the attention weights as a form of explanation [3, 11, 33, 47], however recent works have highlighted how this kind of explanation might lack consistency ”
WiFi CSI Based Passive Human Activity Recognition Using Attention Based BLSTM (2019, background): “Other types of attention models can be found in [26], [27].”