Personalized Cross-Silo Federated Learning on Non-IID Data

By Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, Yong Zhang. AAAI Conference on Artificial Intelligence, 2020

856 citations recorded by the source corpus.

Non-IID data present a tough challenge for federated learning. In this paper, we explore a novel idea of facilitating pairwise collaborations between clients with similar data. We propose FedAMP, a new method employing federated attentive message passing to facilitate similar clients to collaborate more. We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Our extensive experiments on benchmark data sets demonstrate the superior performance of the proposed methods.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/679237737ab2392a87a1f3c44d62b2e37f36bf01

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Bo Du
3. Mang Ye
4. Qiang Yang
5. Bingsheng He
6. Qinbin Li
7. Han Yu
8. Pong Chi Yuen

Citations in the corpus

• Towards Personalized Federated Learning (2021, methodology): “As such, there are PFL works that use distribution-based evaluation frameworks such as histogram profiling [61, 87], variance metrics [37, 55, 88] and metrics at the individual client level [24, 43] to evaluate the performance of personalized models.”
• Model-Contrastive Federated Learning (2021, background): “Another research direction is personalized federated learning [8, 7, 10, 47, 17], which tries to learn personalized local models for each party.”
• Federated Multi-Task Learning under a Mixture of Distributions (2021, background): “Other MTL-based approaches [24, 23, 16, 26, 36] are able to train more general models at the cost of considering simpler penalization terms (e.”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “FedAMP [90] builds a positive feedback loop, iteratively promotes similar client models to collaborate more strongly than dissimilar client models, and adaptively groups similar clients to promote effective collaboration.”
• A Robust Privacy-Preserving Federated Learning Model Against Model Poisoning Attacks (2024, methodology): “This method is in alignment with the methodologies utilized in seminal studies [45], [46], and as observed in the ShieldFL [6].”
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning (2022, methodology): “However, with statistical heterogeneity in FL, it is hard to obtain a single global model which generalizes well to each client (Kairouz et al. 2019; Huang et al. 2021; T Dinh, Tran, and Nguyen 2020).”
• Personalized Federated Learning using Hypernetworks (2021, methodology): “Other approaches to train the global and local models under different regularization (Huang et al., 2020).”