LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets

By Ang Li, Jingwei Sun, Binghui Wang, Lin Duan, Sicheng Li, Yiran Chen, Hai Helen Li. 2020

153 citations recorded by the source corpus.

Federated learning is a popular distributed machine learning paradigm with enhanced privacy. Its primary goal is learning a global model that offers good performance for the participants as many as possible. The technology is rapidly advancing with many unsolved challenges, among which statistical heterogeneity (i.e., non-IID) and communication efficiency are two critical ones that hinder the development of federated learning. In this work, we propose LotteryFL -- a personalized and communication-efficient federated learning framework via exploiting the Lottery Ticket hypothesis. In LotteryFL, each client learns a lottery ticket network (i.e., a subnetwork of the base model) by applying the Lottery Ticket hypothesis, and only these lottery networks will be communicated between the server and clients. Rather than learning a shared global model in classic federated learning, each client learns a personalized model via LotteryFL; the communication cost can be significantly reduced due to the compact size of lottery networks. To support the training and evaluation of our framework, we construct non-IID datasets based on MNIST, CIFAR-10 and EMNIST by taking feature distribution skew, label distribution skew and quantity skew into consideration. Experiments on these non-IID datasets demonstrate that LotteryFL significantly outperforms existing solutions in terms of personalization and communication cost.

Subfields: Artificial Intelligence, Computer Science Applications, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Sociology and Political Science

Carole-Jean Wu

Citations in the corpus

Adaptive Personalized Federated Learning (2020, background): “Due to inherent diversity among local data shards and highly non-IID distribution of the data across clients, FedAvg is hugely sensitive to its hyperparameters, and as a result, does not benefit from a favorable convergence guarantee (Haddadpour and Mahdavi, 2019; Li et al., 2020).”
Model Pruning Enables Efficient Federated Learning on Edge Devices (2019, methodology): “An approach of parameter averaging using structured, sketched, and quantized updates was introduced in [7], which belongs to the broader area of gradient compression/sparsification [28]–[33].”
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning (2021, methodology): “To improve data privacy for ML training, federated learning (FL) is introduced by allowing devices at the edge, such as smartphones, to collaboratively train a shared ML model while keeping all user data locally on the device [11, 34, 52, 64, 71, 77, 84, 116, 120, 135, 137].”