Asynchronous Online Federated Learning for Edge Devices

By Yujing Chen, Yue Ning, Huzefa Rangwala. 2019

57 citations recorded by the source corpus.

Federated learning (FL) is a machine learning paradigm where a shared central model is learned across multiple distributed client devices while the training data remains on edge devices or local clients. Most prior work on federated learning uses Federated Averaging (FedAvg) as an optimization method for training in a synchronized fashion. This involves independent training at multiple edge devices with synchronous aggregation steps. However, the assumptions made by FedAvg are not realistic given the heterogeneity of devices. In particular, the volume and distribution of collected data vary in the training process due to different sampling rates of edge devices. The edge devices themselves also vary in their available communication bandwidth and system configurations, such as memory, processor speed, and power requirements. This leads to vastly different training times as well as model/data transfer times. Furthermore, availability issues at edge devices can lead to a lack of contribution from specific edge devices to the federated model. In this paper, we present an Asynchronous Online Federated Learning (ASO- fed) framework, where the edge devices perform online learning with continuous streaming local data and a central server aggregates model parameters from local clients. Our framework updates the central model in an asynchronous manner to tackle the challenges associated with both varying computational loads at heterogeneous edge devices and edge devices that lag behind or dropout. Experiments on three real-world datasets show the effectiveness of ASO-fed on lowering the overall training cost and maintaining good prediction performance.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sung Ju Hwang
2. Jaehong Yoon
3. Eunho Yang
4. Giwoong Lee
5. Azzam Mourad
6. Wonyong Jeong
7. Tarik Taleb
8. Hadi Otrok

In Computer Science Applications

1. Jaehong Yoon
2. Azzam Mourad
3. Tarik Taleb
4. Hadi Otrok
5. Eunho Yang
6. Yuxi Fan
7. Li Li
8. Omar Abdel Wahab

Citations in the corpus

• Federated Continual Learning with Weighted Inter-client Transfer (2020, background): “TWAFL (Chen et al., 2019) tackles this problem by performing layer-wise parameter aggregation, where shallow layers are aggregated at every step, but deep layers are aggregated in the last few steps of a loop.”
• A Survey on federated learning* (2020, methodology): “updates central model in an asynchronous way by introducing dynamic learning step size [60].”
• Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “The authors of [104] address two essential challenges”
• Federated Semi-Supervised Learning with Inter-Client Consistency (2020, background): “Reliability-based Weighted Knowledge Aggregation Under federated learning frameworks [11, 2, 12, 3, 4], the models could be trained with heterogeneous data distributions with varying number of samples, and not all models at clients may be equally reliable, especially since they learn on unlabeled da”