Split learning for health: Distributed deep learning without sharing raw patient data
By Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, Ramesh Raskar. 2018
964 citations recorded by the source corpus.
Can health entities collaboratively train deep learning models without sharing sensitive raw data? This paper proposes several configurations of a distributed deep learning method called SplitNN to facilitate such collaborations. SplitNN does not share raw data or model details with collaborating institutions. The proposed configurations of splitNN cater to practical settings of i) entities holding different modalities of patient data, ii) centralized and local health entities collaborating on multiple tasks and iii) learning without sharing labels. We compare performance and resource efficiency trade-offs of splitNN and other distributed deep learning methods like federated learning, large batch synchronous stochastic gradient descent and show highly encouraging results for splitNN.
Subfields: Artificial Intelligence, Health Informatics, Federated & Distributed Learning, Privacy & Security
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Qiang Yang
- Medhi Bennis
- Jihong Park
- Murali Annavaram
- Chaoyang He
- Amir Salman Avestimehr
- Yang Liu
- Praneeth Vepakomma
In Health Informatics
- Ramesh Raskar
- Praneeth Vepakomma
- Chaoyang He
- Murali Annavaram
- Vivek Sharma
- Otkrist Gupta
- Chris J. M. Yoon
- Ala Al-Fuqaha
In Federated & Distributed Learning
- Murali Annavaram
- Chaoyang He
- Qiang Yang
- Medhi Bennis
- Praneeth Vepakomma
- Jihong Park
- Yang Liu
- Ramesh Raskar
In Privacy & Security
- Ramesh Raskar
- Praneeth Vepakomma
- Dusist Niyato
- Vivek Sharma
- Chaoyang He
- Murali Annavaram
- Zehui Xiong
- Qiang Yang
Citations in the corpus
- Advances and Open Problems in Federated Learning (2019, neutral): “In contrast with the previous settings which focus on data partitioning and communication patterns, the key idea behind split learning [190, 393]3 is to split the execution of a model on a per-layer basis between the clients and the server.”
- DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks (2020, background): “For the former, distributed learning techniques such as federated learning [6, 7] and split learning [8, 9] are used, where clients communicate with a centralized server using weights and activations and the latter relies on homomorphic encryption [10, 11] and secure enclaves [12, 13].”
- NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, background): “1) Split Learning: This attack model is highly relevant to a popular resource and communication efficient variant of distributed deep learning called split learning [17, 18, 19].”
- SplitNN-driven Vertical Partitioning (2020, methodology): “Furthermore, using SplitNN (Gupta & Raskar, 2018; Vepakomma et al. , 2018b; Sharma et al. , 2019a) also enables the use of distributed sources of data, which results in improved and generalised robust models.”
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “Split learning is computing and memory-efficient variant of FL introduced in [41, 42] where the model is split at a layer and the parts of the model preceding and suc-ceeding this layer are shared across the worker and server, respectively.”
- ExpertMatcher: Automating ML Model Selection for Clients using Hidden Representations (2019, methodology): “…and we experimentally show that we can achieve similar performance as Sharma et al. (2019) to match the task (coarseLevel) assignment and reasonably good performance for class (fine-grained) assignment in a Split Learning distributed computation framework (Vepakomma et al. , 2018b, 2019, 2018a).”
- Detailed comparison of communication efficiency of split learning and federated learning (2019, background): “Federated learning (McMahan et al., 2016; Koneˇcn`y et al., 2016) and Split learning (Gupta & Raskar, 2018; Vepakomma et al., 2018, 2019) are two methods which allow to train a model collectively from various distributed data sources without sharing raw data.”
- Wireless Network Intelligence at the Edge (2018, background): “More general model split-based edge ML frameworks are discussed in [206] and [207], in the context of health applications with private patient data.”
- Group Knowledge Transfer: Collaborative Training of Large CNNs on the Edge (2020, methodology): “Model parallelism-based split learning [12, 13], as shown in Fig.”
- An overview of deep learning in medical imaging focusing on MRI (2018, background): “However, the general deep learning community are focusing a lot of attention on the issue of privacy, and new techniques and frameworks for federated learning [333]50, split learning [334, 335] and differential privacy [336, 337, 338] are rapidly improving.”
- Orthogonal Gradient Descent for Continual Learning (2019, background): “…previous knowledge (i.e. through the model prediction gradients not the actual data) is also related differential private learning (Wu et al., 2017; Li et al., 2018; Pihur et al., 2018; Han et al., 2018) and federated learning (Bonawitz et al., 2019; Smith et al., 2017; Vepakomma et al., 2018).”
- Incentive Mechanism Design for Resource Sharing in Collaborative Edge Learning (2020, positive): “Recently, split learning (SL) has been proposed in [8] whereby each participant only trains the neural network up to a predefined portion, i.e., the cut layer (Fig.”
- SplitFed: When Federated Learning Meets Split Learning (2020, methodology): “Split learning (Vepakomma et al. 2018; Gupta and Raskar 2018) splits a deep learning network W into multiple portions, and these portions are processed and computed on different devices.”
- Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, background): “MIT Media Lab established a split learning project to allow health entities collaboration for training patient diagnostic models without sharing sensitive raw data [316].”
- A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “Split learning is a distributed learning concept, which enables collaborative machine learning without the exchange of datasets with a central server [67, 177].”
- Towards Personalized Federated Learning (2021, background): “As the idea of parameter decoupling has some similarities to split learning (SL) [49, 50], a distributed and private machine learning paradigm, we briefly discuss their differences in this section.”
- Federated Learning for Healthcare Informatics (2019, positive): “[103] built several configurations upon a distributed deep learning method called SplitNN [36] to facilitate the health entities collaboratively training deep learning models without sharing sensitive raw data or model details.”