BEHRT: Transformer for Electronic Health Records

By Yikuan Li, Shishir Rao, José Roberto Ayala Solares, A. Hassaine, R. Ramakrishnan, Dexter Canoy, Yajie Zhu, Kazem Rahimi, Gholamreza Salimi-Khorshidi. Scientific Reports, 2019

679 citations recorded by the source corpus.

Today, despite decades of developments in medicine and the growing interest in precision healthcare, vast majority of diagnoses happen once patients begin to show noticeable signs of illness. Early indication and detection of diseases, however, can provide patients and carers with the chance of early intervention, better disease management, and efficient allocation of healthcare resources. The latest developments in machine learning (including deep learning) provides a great opportunity to address this unmet need. In this study, we introduce BEHRT: A deep neural sequence transduction model for electronic health records (EHR), capable of simultaneously predicting the likelihood of 301 conditions in one’s future visits. When trained and evaluated on the data from nearly 1.6 million individuals, BEHRT shows a striking improvement of 8.0–13.2% (in terms of average precision scores for different tasks), over the existing state-of-the-art deep EHR models. In addition to its scalability and superior accuracy, BEHRT enables personalised interpretation of its predictions; its flexible architecture enables it to incorporate multiple heterogeneous concepts (e.g., diagnosis, medication, measurements, and more) to further improve the accuracy of its predictions; its (pre-)training results in disease and patient representations can be useful for future studies (i.e., transfer learning).

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/57633ff5c6f0708be25e651f51eef29d2fbfe48b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, neutral): “For such reason, we list detailed data usages and access links of each LLM in Section 3.2.”
Multimodal Learning With Transformers: A Survey (2022, background): “, lip reading [77], sign language translation [28], [29]), human-computer interaction [78], healthcare AI [79], [80], surveillance AI [81], etc.”
Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer (2019, methodology): “There are recent works that use Transformer on medical records (Song et al. 2018; Wang and Xu 2019; Shang et al. 2019a; Li et al. 2019), but they either simply replace RNN with Transformer to handle ICU records, or directly apply BERT learning objective on medical records, and do not utilize the…”
Natural Language Processing in Electronic Health Records in relation to healthcare decision-making: A systematic review (2023, methodology): “CPRD [56] Clinical Narratives χ χ χ χ χ EHRs [94] Medication Administration Record χ χ χ χ χ”
SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records (2021, background): “Most of research endeavors have been devoted to developing more advanced deep models or trying to incorporate more data to capture the complexity of diseases and the EHR data, including but not limited to RNNs [5], [8], transformers [9], reverse distillation [10], variational inference [11], deep fe”
Shifting machine learning for healthcare from development to deployment and from models to data (2022, methodology): “When used to predict the likelihood of 301 conditions in future visits, BEHRT achieved an 8–13.2% improvement over the existing state-of-the-art EHR model 152 .”
Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction (2020, result): “To the best of our knowledge, there are only two relevant studies in literature: BEHRT [43] and G-BERT [44].”
Traj-CoA: Patient Trajectory Modeling via Chain-of-Agents for Lung Cancer Risk Prediction (2025, unknown): “Conventional approaches rely on task-specific feature engineering for specialized models, including recurrent neural networks [4, 5, 21], neural differential equations [6, 22, 23], and encoder-based transformers [24, 25, 26, 27].”