SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records

By Chengxi Zang, Fei Wang. 2021

26 citations recorded by the source corpus.

Contrastive learning has demonstrated promising performance in image and text domains either in a self-supervised or a supervised manner. In this work, we extend the supervised contrastive learning framework to clinical risk prediction problems based on longitudinal electronic health records (EHR). We propose a general supervised contrastive loss $\mathcal{L}$Contrastive Cross Entropy+$\lambda \mathcal{L}$Supervised Contrastive Regularizer for learning both binary classification (e.g. in-hospital mortality prediction) and multi-label classification (e.g. phenotyping) in a unified framework. Our supervised contrastive loss practices the key idea of contrastive learning, namely, pulling similar samples closer and pushing dissimilar ones apart from each other, simultaneously by its two components: $\mathcal{L}$Contrastive Cross Entropy tries to contrast samples with learned anchors which represent positive and negative clusters, and $\mathcal{L}$Supervised Contrastive Regularizer tries to contrast samples with each other according to their supervised labels. We propose two versions of the above supervised contrastive loss and our experiments on real-world EHR data demonstrate that our proposed loss functions show benefits in improving the performance of strong baselines and even state-of-the-art models on benchmarking tasks for clinical risk predictions. Our loss functions work well with extremely imbalanced data which are common for clinical risk prediction problems. Our loss functions can be easily used to replace (binary or multi-label) cross-entropy loss adopted in existing clinical predictive models. The Pytorch code is released at https://github.com/calvin-zcx/SCEHR.

Subfields: Artificial Intelligence, Health Information Management, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Radiology, Nuclear Medicine and Imaging

Curtis P. Langlotz

In Representation & Self-Supervised Learning

Citations in the corpus

Contrastive Learning of Medical Visual Representations from Paired Images and Text (2020, methodology): “Since the release of our work, similar image-text pretraining strategies have been used to improve more downstream healthcare tasks including image regeneration (Wang et al., 2021), medical visual question answering (Eslami et al., 2021) and clinical risk prediction (Zang and Wang, 2021), etc.”