SplitFed: When Federated Learning Meets Split Learning

By Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Ahmet Camtepe. 2020

923 citations recorded by the source corpus.

Federated learning (FL) and split learning (SL) are two popular distributed machine learning approaches. Both follow a model-to-data scenario; clients train and test machine learning models without sharing raw data. SL provides better model privacy than FL due to the machine learning model architecture split between clients and the server. Moreover, the split model makes SL a better option for resource-constrained environments. However, SL performs slower than FL due to the relay-based training across multiple clients. In this regard, this paper presents a novel approach, named splitfed learning (SFL), that amalgamates the two approaches eliminating their inherent drawbacks, along with a refined architectural configuration incorporating differential privacy and PixelDP to enhance data privacy and model robustness. Our analysis and empirical results demonstrate that (pure) SFL provides similar test accuracy and communication efficiency as SL while significantly decreasing its computation time per global epoch than in SL for multiple clients. Furthermore, as in SL, its communication efficiency over FL improves with the number of clients. Besides, the performance of SFL with privacy and robustness measures is further evaluated under extended experimental settings.

Subfields: Artificial Intelligence, Computer Science Applications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Electrical and Electronic Engineering

Citations in the corpus

NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, methodology): “Split learning has been adopted in Internet of Things (IOT) and edgedevice machine learning settings in [20, 21, 22, 23] including multi-modal fusion based machine learning across edge devices in [24, 25] with data collected at the edge on imagery and millimeter wave (mmWave) radio frequency (RF) si”
HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients (2020, methodology): “To reduce communication costs in FL, some studies propose to use data compression techniques such as quantization and sketching (Konečnỳ et al., 2016; Alistarh et al., 2017; Ivkin et al., 2019), and some propose to adopt split learning (Thapa et al., 2020).”
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things (2020, methodology): “Besides, to reduce the training overhead due to sequential training in SplitNN, a parallel machine learning model update paradigm of FL is applicable in its client-side section [31].”
Towards Personalized Federated Learning (2021, background): “SL therefore has a privacy advantage over FL as the server and clients do not have full access to the global and local models [51].”
Decentralized Federated Learning: A Survey and Perspective (2023, background): “3) Split FL splits the model for learning, where the server is responsible for some model layers [126], [127], [128].”
Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges (2022, methodology): “• Optimize algorithms to suit the heterogeneous node resources and capabilities [27], [97]–[99] Optimization of Federation Communications • Efficient distribution schemes that minimize the number of network exchanges [58], [72], [100]”