TiFL: A Tier-based Federated Learning System

By Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalie Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, Yue Cheng. 2020

370 citations recorded by the source corpus.

Federated Learning (FL) enables learning a shared model acrossmany clients without violating the privacy requirements. One of the key attributes in FL is the heterogeneity that exists in both resource and data due to the differences in computation and communication capacity, as well as the quantity and content of data among different clients. We conduct a case study to show that heterogeneity in resource and data has a significant impact on training time and model accuracy in conventional FL systems. To this end, we propose TiFL, a Tier-based Federated Learning System, which divides clients into tiers based on their training performance and selects clients from the same tier in each training round to mitigate the straggler problem caused by heterogeneity in resource anddata quantity. To further tame the heterogeneity caused by non-IID (Independent and Identical Distribution) data and resources, TiFL employs an adaptive tier selection approach to update the tiering on-the-fly based on the observed training performance and accuracy. We prototype TiFL in a FL testbed following Google's FL architecture and evaluate it using the state-of-the-art FL benchmarks. Experimental evaluation shows that TiFL outperforms the conventional FL in various heterogeneous conditions. With the proposed adaptive tier selection policy, we demonstrate that TiFL achieves much faster training performance while achieving the same or better test accuracy across the board.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

Citations in the corpus

IBM Federated Learning: an Enterprise Framework White Paper V0.1 (2020, background): “In the past few years, a variety of algorithms have emerged for training diﬀerent types of ML models via federated learning, e.g., [7, 10, 14, 19, 31, 32, 33, 34, 35].”
Asynchronous Online Federated Learning for Edge Devices with Non-IID Data (2019, background): “Since the server aggregates after all clients finish, the extended period of waiting time in a synchronized optimization protocol will lead to idling and wastage of computing resources [29], [30].”
Asynchronous Online Federated Learning for Edge Devices (2019, background): “Since the server aggregates aer all clients nish, the extended period of waiting time in a synchronized optimization protocol will lead to idling and wastage of computing resources [9, 10].”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “To better deal with the deviation problem caused by statistical, communication and device heterogeneity, TiFL [20] adopts an adaptive layer selection method that divides clients into different layers according to the training time and then selects clients from the same layer in each training round.”
A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, methodology): “[129] proposed a tierbased FL (TiFL) system classifying the clients into tiers based on the performance of their training while applying adaptive tier-based clients selection.”