Federated pretraining and fine tuning of BERT using clinical notes from multiple silos
By Dianbo Liu, Timothy Miller. 2020
38 citations recorded by the source corpus.
Large scale contextual representation models, such as BERT, have significantly advanced natural language processing (NLP) in recently years. However, in certain area like healthcare, accessing diverse large scale text data from multiple institutions is extremely challenging due to privacy and regulatory reasons. In this article, we show that it is possible to both pretrain and fine tune BERT models in a federated manner using clinical texts from different silos without moving the data.
Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Murali Annavaram
- Qiang Yang
- Chaoyang He
- Yang Liu
- S. Avestimehr
- Praneeth Vepakomma
- Abhishek Singh
- Ramesh Raskar
In Radiology, Nuclear Medicine and Imaging
- Praneeth Vepakomma
- Chaoyang He
- Murali Annavaram
- Abhishek Singh
- S. Avestimehr
- Qiang Yang
- Yang Liu
- Peilin Zhao
Citations in the corpus
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “Promising application domains include, but are not limited to, computer vision [77, 78], natural language processing [79, 80, 81, 82, 83], finance [74, 35, 84], transportation [85, 86, 87, 88, 89, 90, 91, 92, 93, 94], digital health [95, 96, 97, 98, 99, 100, 101], recommendation [102, 103, 104, 105,…”