Multi-center federated learning: clients clustering for better personalization

By Guodong Long, Tao Shen, Tianyi Zhou, Xianzhi Wang, Jing Jiang. World wide web (Bussum), 2020

379 citations recorded by the source corpus.

Personalized decision-making can be implemented in a Federated learning (FL) framework that can collaboratively train a decision model by extracting knowledge across intelligent clients, e.g. smartphones or enterprises. FL can mitigate the data privacy risk of collaborative training since it merely collects local gradients from users without access to their data. However, FL is fragile in the presence of statistical heterogeneity that is commonly encountered in personalized decision making, e.g., non-IID data over different clients. Existing FL approaches usually update a single global model to capture the shared knowledge of all users by aggregating their gradients, regardless of the discrepancy between their data distributions. By comparison, a mixture of multiple global models could capture the heterogeneity across various clients if assigning the client to different global models (i.e., centers) in FL. To this end, we propose a novel multi-center aggregation mechanism to cluster clients using their models’ parameters. It learns multiple global models from data as the cluster centers, and simultaneously derives the optimal matching between users and centers. We then formulate it as an optimization problem that can be efficiently solved by a stochastic expectation maximization (EM) algorithm. Experiments on multiple benchmark datasets of FL show that our method outperforms several popular baseline methods. The experimental source codes are publicly available on the Github repository (GitHub repository: https://github.com/mingxuts/multi-center-fed-learning ).

Subfields: Artificial Intelligence, Information Systems, Transportation

View Paper: https://www.semanticscholar.org/paper/3d9d556474da0ee81f3bc189c551721760ce1d68

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Dacheng Tao
3. Jian Liang
4. Guodong Long
5. Chengqi Zhang
6. Jing Jiang
7. Jiashi Feng
8. Lichao Sun

In Information Systems

1. Bo Du
2. Mang Ye
3. Jing Jiang
4. Guodong Long
5. Tianyi Zhou
6. Yue Tan
7. Lu Liu
8. Chengqi Zhang

In Transportation

1. Bo Du
2. Mang Ye
3. Chengqi Zhang
4. Guodong Long
5. Jing Jiang
6. Philip S. Yu
7. Qiang Yang
8. Han Yu

Citations in the corpus

• Towards Personalized Federated Learning (2021, methodology): “Multi-Task Learning [59]–[62] [59], [61]–[62] [61]–[62] [59, 61] - [59, 61] - Model Interpolation [17], [64]–[65] [17], [63]–[65] [63]–[65] [65] [65] [65] - - Clustering [66]–[70], [72] [66]–[70], [72] [66]–[67], [69]–[70], [72] - - -”
• FedProto: Federated Prototype Learning across Heterogeneous Clients (2021, positive): “The local average test accuracy of FedProto is greater than for the FeSEM , FedProx , FedPer and FedAvg algorithms in all the settings.”
• Learn from Others and Be Yourself in Heterogeneous Federated Learning (2022, methodology): “To tackle this challenge, one type of methods typically performs fine-tuning for several rounds [38, 50, 58, 74, 88].”
• Federated Learning from Pre-Trained Models: A Contrastive Learning Approach (2022, methodology): “[62] Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang.”
• Decentralized Federated Learning: A Survey and Perspective (2023, background): “1) Client heterogeneity predominantly stems from three latent factors: 1) individual; 2) group; and 3) systemic heterogeneity [80], [83], [84], [85].”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “[220] leverage a novel multi-center aggregation mechanism to address the problem of statistical heterogeneity.”
• Federated Learning for Open Banking (2021, background): “Xie et al. [34] addresses the non-IID challenge of federated learning and proposes a multi-center aggregation approach.”
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data (2021, background): “There are also heuristic approaches that divide clients into different clusters based on their learning tasks (objectives) and perform aggregation only within the cluster [33, 34, 22, 35].”