Multi-layer Representation Learning for Medical Concepts
By Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James E. Bost, Javier Tejedor-Sojo, Jimeng Sun. 2016
540 citations recorded by the source corpus.
Proper representations of medical concepts such as diagnosis, medication, procedure codes and visits from Electronic Health Records (EHR) has broad applications in healthcare analytics. Patient EHR data consists of a sequence of visits over time, where each visit includes multiple medical concepts, e.g., diagnosis, procedure, and medication codes. This hierarchical structure provides two types of relational information, namely sequential order of visits and co-occurrence of the codes within a visit. In this work, we propose Med2Vec, which not only learns the representations for both medical codes and visits from large EHR datasets with over million visits, but also allows us to interpret the learned representations confirmed positively by clinical experts. In the experiments, Med2Vec shows significant improvement in prediction accuracy in clinical applications compared to baselines such as Skip-gram, GloVe, and stacked autoencoder, while providing clinically meaningful interpretation.
Subfields: Artificial Intelligence, Molecular Biology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Molecular Biology
Citations in the corpus
- RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism (2016, methodology): “MLP has been used for representation learning in EHR data [10].”
- Opportunities and obstacles for deep learning in biology and medicine (2017, methodology): “Techniques have been developed for free text medical notes [77], ICD and National Drug Codes, and claims data [78].”
- Information Retrieval: A Biomedical and Health Perspective (2020, unknown): “In a study of 120 recent publications that used data from the US National Inpatient Sample, the majority of users did not adhere to required practices [225].”
- Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer (2019, background): “…organizations enabled deep learning methods to show impressive performance in diverse tasks such as predicting diagnosis (Lipton et al. 2015; Choi et al. 2016a; Rajkomar et al. 2018), learning medical concept representations (Che et al. 2015; Choi, Chiu, and Sontag 2016; Choi et al. 2016b;…”
- Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models (2018, methodology): “For example, deep learning has been used to learn effective representations for patient records [14, 27] to support disease phenotyping [8] and conduct predictive modeling [9, 31, 34].”
- Mining Electronic Health Records: A Survey (2017, unknown): “…2012] Time Agnostic Case-Control or Cohort Time Aware Case-Control or Cohort [Nachimuthu et al. 2010; Verduijn et al. 2007; Batal et al. 2012; Choi et al. 2016; Perotte and Hripcsak 2013] Cohort Identification Descritive Atemporal [Roque et al. 2011; Bauer-Mehren et al. 2013] Descritive…”
- Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding (2021, background): “However, most of the existing approaches for learning representations are designed for specific downstream tasks and require labeling by experts (Choi et al., 2016b;c; Tonekaboni et al., 2020).”
- Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records (2017, methodology): “For EHR data, researchers attempted different types of neural networks to exploit the natural temporarily, such as recurrent neural networks (RNNs) [12], [13], deep state space model [14], and convolutional neural networks (CNNs) [15], [16].”
- Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks (2017, methodology): “Non-temporal models such as Med2Vec [9] generate easily interpretable low-dimensional representations of the medical codes, but do not account for the temporal nature of the EHR data.”
- Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis (2017, methodology): “DeepPatient Multi-outcome Prediction Miotto [15] Deepr Hospital Re-admission Prediction Nguyen [20] DeepCare EHR Concept Representation Pham [21] Doctor AI Heart Failure Prediction Choi [22] Med2Vec EHR Concept Representation Choi [23] eNRBM Suicide risk stratification Tran [24]”
- Patient Subtyping via Time-Aware LSTM Networks (2017, unknown): “A dierent approach to computational subtyping was introduced in [11].”
- $\mathtt {Deepr}$: A Convolutional Net for Medical Records (2016, background): “Embedding of medical concepts has been proposed in contemporary work [10], [46]–[48].”
- Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review (2018, background): “29 For concept embedding tasks, deep learning models are often trained in an unsupervised setting without target labels.”
- High-Dimension Human Value Representation in Large Language Models (2024, background): “Embeddings have also been applied in healthcare and personalized recommendation systems to model complex behaviors and preferences [69, 70].”