Client-Edge-Cloud Hierarchical Federated Learning

By Lumin Liu, Shenghui Song, Khaled Ben Letaief. 2019

1,027 citations recorded by the source corpus.

Federated Learning is a collaborative machine learning framework to train a deep learning model without accessing clients’ private data. Previous works assume one central parameter server either at the cloud or at the edge. The cloud server can access more data but with excessive communication overhead and long latency, while the edge server enjoys more efficient communications with the clients. To combine their advantages, we propose a client-edge-cloud hierarchical Federated Learning system, supported with a HierFAVG algorithm that allows multiple edge servers to perform partial model aggregation. In this way, the model can be trained faster and better communication-computation trade-offs can be achieved. Convergence analysis is provided for HierFAVG and the effects of key parameters are also investigated, which lead to qualitative design guidelines. Empirical experiments verify the analysis and demonstrate the benefits of this hierarchical architecture in different data distribution scenarios. Particularly, it is shown that by introducing the intermediate edge servers, the model training time and the energy consumption of the end devices can be simultaneously reduced compared to cloud-based Federated Learning.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “As demonstrated in Figure 3, FL has various topology deﬁnitions, such as vertical FL [34, 35, 36, 37, 38, 39, 40], split learning [41, 42], decentralized FL [43, 44, 45, 46], and Hierarchical FL [47, 48, 49, 50, 51, 52].”
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges (2020, methodology): “In [31], a client-edge-cloud hierarchical federated learning framework has been proposed.”
Federated Learning in Edge Computing: A Systematic Survey (2022, background): “Depending on how the global learning model is implemented, there are three types of FL structures [21]: cloud-enabled, edge-enabled, and hierarchical (client-edgecloud-enabled) FL.”
Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “Periodic and dynamic aggregation approaches [22], [25], [58] have been proposed in an attempt to decrease the communication overhead and avoid these bottlenecks.”
Communication-Efficient Edge AI: Algorithms and Systems (2020, methodology): “To reduce the costly communication with the remote cloud, edge server assisted hierarchical federated averaging was proposed in [183].”
A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning (2021, methodology): “that the data sets at different devices or BSs follow different distributions [115], [116], if a global DNN is directly used by a device or a BS for inference, it can neither guarantee the QoS requirements nor achieve optimal resource utilization efﬁciency.”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “Compared with the general federated learning process, the cloud-fog-IoT federated learning introduces the middle layer of the fog server between the cloud server and the IoT device [134].”