Asynchronous Online Federated Learning for Edge Devices with Non-IID Data
By Yujing Chen, Yue Ning, Martin Slawski, Huzefa Rangwala. 2020 IEEE International Conference on Big Data (Big Data), 2019
480 citations recorded by the source corpus.
Federated learning (FL) is a machine learning paradigm where a shared central model is learned across distributed devices while the training data remains on these devices. Federated Averaging (FedAvg) is the leading optimization method for training non-convex models in this setting with a synchronized protocol. However, the assumptions made by FedAvg are not realistic given the heterogeneity of devices. First, the volume and distribution of collected data vary in the training process due to different sampling rates of edge devices. Second, the edge devices themselves also vary in latency and system configurations, such as memory, processor speed, and power requirements. This leads to vastly different computation times. Third, availability issues at edge devices can lead to a lack of contribution from specific edge devices to the federated model. In this paper, we present an Asynchronous Online Federated Learning (ASO-Fed) framework, where the edge devices perform online learning with continuous streaming local data and a central server aggregates model parameters from clients. Our framework updates the central model in an asynchronous manner to tackle the challenges associated with both varying computational loads at heterogeneous edge devices and edge devices that lag behind or dropout. We perform extensive experiments on a benchmark image dataset and three real-world datasets with non-IID streaming data. The results demonstrate ASO-Fed converging fast and maintaining good prediction performance.
Subfields: Artificial Intelligence, Computer Science Applications
View Paper: https://www.semanticscholar.org/paper/d1a7d35ac84502c14737567b72249e952e824d72
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sung Ju Hwang
- Jaehong Yoon
- Eunho Yang
- Giwoong Lee
- Yong Xiang
- Wonyong Jeong
- Michael G. Rabbat
- Hongyuan Zhan
In Computer Science Applications
Citations in the corpus
- Asynchronous Federated Learning on Heterogeneous Devices: A Survey (2021, methodology): “In [60], a decay coefficient is proposed with similar effects, balancing the previous and current models.”
- Federated Continual Learning with Adaptive Parameter Communication (2020, methodology): “Chen et al. (2019b) tackle this problem by performing layer-wise parameter aggregation, where some layers(i.e. shallow layers) are aggregated in every step, but other layers(i.e. deep layers) are aggregated in last a few steps of a loop.”
- Federated Semi-Supervised Learning with Inter-Client Consistency (2020, background): “Federated Learning (FL) (McMahan et al., 2017; Zhao et al., 2018; Li et al., 2018; Chen et al., 2019a;b), in which multiple clients collaboratively learn a global model via coordinated communication, has been an active topic of research over the past few years.”
- Federated Learning with Buffered Asynchronous Aggregation (2021, background): “Most asynchronous FL (AsyncFL) works, such as the works (Xie et al., 2019; van Dijk et al., 2020; Chai et al., 2020; Chen et al., 2019; Wu et al., 2020; Li et al., 2021) have been focused on solving the straggler problem by designing asynchronous FL algorithms.”