Federated learning of predictive models from federated Electronic Health Records
By Theodora S. Brisimi, Ruidi Chen, Theofanie Mela, Alexander Olshevsky, Ioannis Ch. Paschalidis, Wei Shi. 2018
938 citations recorded by the source corpus.
Background In an era of “big data,” computationally efficient and privacy-aware solutions for large-scale machine learning problems become crucial, especially in the healthcare domain, where large amounts of data are stored in different locations and owned by different entities. Past research has been focused on centralized algorithms, which assume the existence of a central data repository (database) which stores and can process the data from all participants. Such an architecture, however, can be impractical when data are not centrally located, it does not scale well to very large datasets, and introduces single-point of failure risks which could compromise the integrity and privacy of the data. Given scores of data widely spread across hospitals/individuals, a decentralized computationally scalable methodology is very much in need. Objective We aim at solving a binary supervised classification problem to predict hospitalizations for cardiac events using a distributed algorithm. We seek to develop a general decentralized optimization framework enabling multiple data holders to collaborate and converge to a common predictive model, without explicitly exchanging raw data. Methods We focus on the soft-margin l1-regularized sparse Support Vector Machine (sSVM) classifier. We develop an iterative cluster Primal Dual Splitting (cPDS) algorithm for solving the large-scale sSVM problem in a decentralized fashion. Such a distributed learning scheme is relevant for multi-institutional collaborations or peer-to-peer applications, allowing the data holders to collaborate, while keeping every participant’s data private. Results We test cPDS on the problem of predicting hospitalizations due to heart diseases within a calendar year based on information in the patients Electronic Health Records prior to that year. cPDS converges faster than centralized methods at the cost of some communication between agents. It also converges faster and with less communication overhead compared to an alternative distributed algorithm. In both cases, it achieves similar prediction accuracy measured by the Area Under the Receiver Operating Characteristic Curve (AUC) of the classifier. We extract important features discovered by the algorithm that are predictive of future hospitalizations, thus providing a way to interpret the classification results and inform prevention efforts.
Subfields: Artificial Intelligence, Health Information Management, Federated & Distributed Learning, Medical & Health AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Qiang Yang
- Peter Richtárik
- Jakub Konecný
- Martin Jaggi
- Dacheng Tao
- Virginia Smith
- Sebastian U. Stich
- Yang Liu
In Health Information Management
- Arne Küderle
- Mike Tse
- Muhammad Bilal
- Ala Al-Fuqaha
- Rodolfo Stoffel Antunes
- Aruna Prasad Seneviratne
- Cristiano André da Costa
- Ming Ding
In Federated & Distributed Learning
- Virginia Smith
- Jakub Konecný
- Qiang Yang
- Bingsheng He
- Qinbin Li
- Tian Li
- Zachary B. Charles
- Sebastian U. Stich
In Medical & Health AI
- Arne Küderle
- Mike Tse
- Dacheng Tao
- Rodolfo Stoffel Antunes
- Cristiano André da Costa
- Jing Zhang
- Qiang Yang
- Fei Wang
Citations in the corpus
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “Several current applications of FL include text prediction in mobile device messaging [1, 2, 3, 4, 5], speech recognition [6], face recognition for device access [7, 8], and maintaining decentralized predictive models across health organizations [9, 10, 11].”
- Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy (2020, background): “• When information is located in data silos, for instance, healthcare industry is usually reluctant to disclose their information records, and this is kept as sequestered data [4].”
- On the Outsized Importance of Learning Rates in Local Update Methods (2020, background): “Federated learning has enabled privacy-aware learning in a variety of applications (Hard et al., 2018; Chen et al., 2019; Brisimi et al., 2018; Samarakoon et al., 2018; Hard et al., 2020), and has seen a large volume of work on the intersection of federated learning with topics including…”
- Three Approaches for Personalization with Applications to Federated Learning (2020, positive): “This global model benefits from having access to client data and can often perform better on several learning problems, including next word prediction (Hard et al., 2018; Yang et al., 2018) and predictive models in health (Brisimi et al., 2018).”
- SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning (2019, background): “…(Hard et al., 2018; Yang et al., 2018), emoji prediction (Ramaswamy et al., 2019), decoder models (Chen et al., 2019b), vocabulary estimation (Chen et al., 2019a), low latency vehicle-to-vehicle communication (Samarakoon et al., 2018), and predictive models in health (Brisimi et al., 2018).”
- Towards Federated Learning at Scale: System Design (2019, methodology): “In addition, Federated Learning has already been proposed in the context of vehicle-to-vehicle communication (Sama-rakoon et al., 2018) and medical applications (Brisimi et al., 2018).”
- Federated Learning: Challenges, Methods, and Future Directions (2019, methodology): “Federated learning methods have been deployed in practice by major companies [5], [41] and play a critical role in supporting privacy-sensitive applications where training data are distributed at the edge [8], [19].”
- SCAFFOLD: Stochastic Controlled Averaging for Federated Learning (2019, background): “, 2018), and predictive models in health (Brisimi et al., 2018).”
- Algorithm fairness in artificial intelligence for medicine and healthcare (2023, background): “…learning has been applied to a variety of clinical settings to overcome data-interoperability standards that would usually prohibit sensitive health data from being shared and to tackle low-data regimes of clinical machine-learning tasks, for example for the prediction of rare diseases 98,206–220 .”
- From distributed machine learning to federated learning: a survey (2021, background): “, healthcare [18], computer visions [86], and industrial groups,”
- Federated learning for predicting clinical outcomes in patients with COVID-19 (2021, background): “Governance of data for FL is maintained locally, alleviating privacy concerns, with only model weights or gradients communicated between client sites and the federated server 38,39 .”
- A Survey on federated learning* (2020, background): “Furthermore, it can be applied to handle information dispersed in various institutions such as hospitals or clinics to solve the problem about detached islands while keeping data on local database [56].”
- Federated Learning for Internet of Things: A Comprehensive Survey (2021, methodology): “[114] EHRs management DFL SVM Health users Data center An FL-based distributed binary supervised classification scheme.”
- A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent (2019, background): “Problems in the form of (1) find applications in multiagent target seeking [1], distributed machine learning [2]–[8], and wireless networks [5], [9], [10], among other scenarios.”
- Shifting machine learning for healthcare from development to deployment and from models to data (2022, methodology): “It has been used to leverage EHRs to train models to predict hospitalization due to heart disease 72 , to promote the development of ‘digital twins’ or ‘Google for patients’ 73 , and to develop a Coronavirus disease 2019 (COVID-19) chest-CT lesion segmenter 74 .”
- A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, methodology): “Each hospital shares the model trained with data from their patients and gets the global model for diagnosis [28].”
- Federated Learning for Healthcare Informatics (2019, methodology): “Mortality Prediction [37] Autoencoder 5-50 eICU Collaborative Research Database [73] Hospitalizations Prediction [9] SVM 5, 10 Boston Medical Center Preterm-birth Prediction [8] RNN 50 Cerner Health Facts”
- Federated Learning for Healthcare: Systematic Review and Architecture Proposal (2022, methodology): “[44], which explores the design of a distributed architecture to solve large-scale sparse support vector machine (SVM) models.”