End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

By Yansong Gao, Minki Kim, Sharif Abuadbba, Yeonjae Kim, Chandra Thapa, Kyuyeon Kim, Seyit Ahmet Camtepe, Hyoungshick Kim, Surya Nepal. 2020

244 citations recorded by the source corpus.

Federated learning (FL) and split neural networks (SplitNN) are state-of-art distributed machine learning techniques to enable machine learning without directly accessing raw data on clients or end devices. In theory, such distributed machine learning techniques have great potential in distributed applications, in which data are typically generated and collected at the client-side while the collected data should be processed by the application deployed at the server-side. However, there is still a significant gap in evaluating the performance of those techniques concerning their practicality in the Internet of Things (IoT)-enabled distributed systems constituted by resource-constrained devices. This work is the first attempt to provide empirical comparisons of FL and SplitNN in real-world IoT settings in terms of learning performance and device implementation overhead. We consider a variety of datasets, different model architectures, multiple clients, and various performance metrics. For the learning performance (i.e., model accuracy and convergence time), we empirically evaluate both FL and SplitNN under different types of data distributions such as imbalanced and non-independent and identically distributed (non-IID) data. We show that the learning performance of SplitNN is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. For implementation overhead, we mount both FL and SplitNN on Raspberry Pi devices and comprehensively evaluate their overhead, including training time, communication overhead, power consumption, and memory usage. Our key observations are that under the IoT scenario where the communication traffic is the primary concern, FL appears to perform better over SplitNN because FL has a significantly lower communication overhead compared with SplitNN. However, our experimental results also demonstrate that neither FL or SplitNN can be applied to a heavy model, e.g., with several million parameters, on resource-constrained IoT devices because its training cost would be too expensive for such devices. Source code is released and available: https://github.com/Minki-Kim95/Federated-Learning-and-Split-Learning-with-raspberry-pi.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, background): “Split learning has been adopted in Internet of Things (IOT) and edge-device machine learning settings in [20, 21, 22, 23] including multi-modal fusion based machine learning across edge devices in [24, 25] with data collected at the edge on imagery and millimeter wave (mmWave) radio frequency (RF)…”
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, unknown): “In [79], it has been experimentally verified that SL achieves better accuracy and faster convergence than FL when data distributions at multiple clients are imbalanced.”
SplitFed: When Federated Learning Meets Split Learning (2020, background): “In contrast, FL is preferred if the amount of data samples is increased by keeping the number of clients or model size low [25,26].”
Towards Personalized Federated Learning (2021, background): “SL also performs worse than FL on non-IID data and has higher communication overheads [52].”