Survey of Personalization Techniques for Federated Learning

By Viraj Kulkarni, Milind Kulkarni, Aniruddha Pant. 2020

394 citations recorded by the source corpus.

Federated learning enables machine learning models to learn from private decentralized data without compromising privacy. The standard formulation of federated learning produces one shared model for all clients. Statistical heterogeneity due to non-IID distribution of data across devices often leads to scenarios where, for some clients, the local models trained solely on their private data perform better than the global shared model thus taking away their incentive to participate in the process. Several techniques have been proposed to personalize global models to work better for individual clients. This paper highlights the need for personalization and surveys recent research on this topic.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Peter Richtárik
2. Filip Hanzely
3. Samuel Horváth
4. Dacheng Tao
5. Qiang Yang
6. Fei Wang
7. Dusist Niyato
8. Ji Liu

In Computer Science Applications

1. Jiawen Kang
2. Li-zhen Cui
3. Dacheng Tao
4. Jian Pei
5. Bo Du
6. Qiang Yang
7. Dusist Niyato
8. Dejing Dou

Citations in the corpus

• Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges (2021, background): “A number of possible research directions are also highlighted.”
• Personalized Federated Learning: An Attentive Collaboration Approach (2020, background): “Similarly, some meta learning methods [9, 15, 16, 23, 24, 25] can be extended to build personalized models by adapting a trained global model on the local data of a client with a few fine-tuning steps [8].”
• FedBABU: Towards Enhanced Representation for Federated Image Classification (2021, background): “In addition, various technologies and algorithms for personalized FL have been presented and will be helpful to read; see Tan et al. (2021) and Kulkarni et al. (2020) for more details.”
• Towards Personalized Federated Learning (2021, background): “There has been a short survey [Kulkarni et al., 2020] that provides a brief overview of PFL.”
• Personalized Federated Learning using Hypernetworks (2021, background): “To overcome these issues, Personalized Federated Learning (PFL) aims to personalize the global model for each client in the federation (Kulkarni et al., 2020).”
• Lower Bounds and Optimal Algorithms for Personalized Federated Learning (2020, background): “See also [25, 22] for a survey on personalized FL.”
• Federated Learning for Healthcare Informatics (2019, background): “, personalization techniques [59], semi-supervised learning algorithms [49], threat models [68], and mobile edge networks [66].”
• From distributed machine learning to federated learning: a survey (2021, background): “The primary incentive for clients to participate in federated learning is obtaining better models [87], while the benefit of participating in federated learning for clients who have sufficient private data to train accurate local models is disputable.”
• Personalized Cross-Silo Federated Learning on Non-IID Data (2020, methodology): “Similarly, meta-learning methods (Chen et al. 2018; Fallah, Mokhtari, and Ozdaglar 2020; Jiang et al. 2019; Khodak, Balcan, and Talwalkar 2019; Kulkarni, Kulkarni, and Pant 2020; Nichol, Achiam, and Schulman 2018) can be extended to customize personalized models by adapting a well-trained global…”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “[106] They point out that statistical heterogeneity can hinder federated learning and highlight the need for personalization.”
• Federated learning for 6G communications: Challenges, methods, and future directions (2020, background): “Prior studies [43]–[46] adopt different personalized techniques to provide users with real-time personalized services, which is a solid step towards personalized FL.”