A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection

By Qinbin Li, Zeyi Wen, Zhaomin Wu, Bingsheng He. 2019

1,461 citations recorded by the source corpus.

As data privacy increasingly becomes a critical societal concern, federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing systems and infrastructures to ease the development of various federated learning algorithms. Similar to deep learning systems such as PyTorch and TensorFlow that boost the development of deep learning, federated learning systems (FLSs) are equivalently important, and face challenges from various aspects such as effectiveness, efficiency, and privacy. In this survey, we conduct a comprehensive review on federated learning systems. To understand the key design system components and guide future research, we introduce the definition of federated learning systems and analyze the system components. Moreover, we provide a thorough categorization for federated learning systems according to six different aspects, including data distribution, machine learning model, privacy mechanism, communication architecture, scale of federation and motivation of federation. The categorization can help the design of federated learning systems as shown in our case studies. By systematically summarizing the existing federated learning systems, we present the design factors, case studies, and future research opportunities.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bo Du
2. Dacheng Tao
3. Mang Ye
4. Philip S. Yu
5. Lichao Sun
6. Bingsheng He
7. Qinbin Li
8. Hangyu Zhu

Citations in the corpus

• Machine Learning for Synthetic Data Generation: a Review (2023, background): “In these cases, generating synthetic data can be very useful, and we will investigate ways of creating anonymized datasets with differential privacy protections [8].”
• Federated Learning on Non-IID Data: A Survey (2021, background): “Analyses of threats and additional privacy preservation techniques in FL are presented in [128, 81, 8, 92].”
• Model-Contrastive Federated Learning (2021, background): “To address the above challenge, federated learning [20, 44, 27, 26] enables multiple parties to jointly learn a machine learning model without exchanging their local data.”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “[119] They provide a comprehensive analysis on federated learning from systems perspective, including system components, taxonomy, summary, design, and vision.”
• From distributed machine learning to federated learning: a survey (2021, background): “Some surveys [89,114,129] focus on the data privacy and security of FL.”
• Federated learning for medical image analysis: A survey (2023, background): “More comparative reviews and evaluations can be found in (Kholod et al., 2020; Li et al., 2021).”
• A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “These three surveys [97, 99, 105] emphasized the efficiency and effectiveness of FL instead of privacy preservation.”
• Decentralized Federated Learning: A Survey and Perspective (2023, background): “…values indicating the relative strength of the respective FL approach in that domain. metrics, which are the focal points of current state-of-the-art (SOTA) research and worthy of further investigation [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48].”
• Vertical Federated Learning: Concepts, Advances, and Challenges (2022, background): “Companies and institutions owning only small and fragmented data have constantly been looking for compensating data partners to collaboratively develop artificial intelligence technology for maximizing data utilization [7], [8].”
• Federated Learning on Non-IID Data Silos: An Experimental Study (2021, methodology): “A key challenge in FL is the non-IID data among the parties [29, 37].”
• Federated Learning in Edge Computing: A Systematic Survey (2022, methodology): “[82] Proposed an FL building block taxonomy with six different aspects: data distribution, ML model, privacy mechanism, communication architecture, the scale of the federation, and the motivation for FL X X X X X X”
• Federated Learning for Internet of Things: A Comprehensive Survey (2021, methodology): “- Networking structure: From the networking perspective, this category can be divided into two small classes, including centralized FL and decentralized FL [11] as summarized in Fig.”
• Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “In [12], the authors classify the federated learning approaches based on six aspects,”
• Blockchain-Federated-Learning and Deep Learning Models for COVID-19 Detection Using CT Imaging (2020, background): “The previous studies do not consider to share the medical data without compromising the privacy of organizations[38], [39].”