Towards Efficient and Privacy-Preserving Federated Deep Learning

By Meng Hao, Hongwei Li, Guowen Xu, Sen Liu, Haomiao Yang. 2019

200 citations recorded by the source corpus.

Deep learning has been applied in many areas, such as computer vision, natural language processing and emotion analysis. Differing from the traditional deep learning that collects users' data centrally, federated deep learning requires participants to train the networks on private datasets and share the training results, and hence has more gratifying efficiency and stronger security. However, it still presents some privacy issues since adversaries can deduce users' privacy from local outputs, such as gradients. While the problem of private federated deep learning has been an active research issue, the latest research findings are still inadequate in terms of security, accuracy and efficiency. In this paper, we propose an efficient and privacy-preserving federated deep learning protocol based on stochastic gradient descent method by integrating the additively homomorphic encryption with differential privacy. Specifically, users add noises to each local gradients before encrypting them to obtain the optical performance and security. Moreover, our scheme is secure to honest-but-curious server setting even if the cloud server colludes with multiple users. Besides, our scheme supports federated learning for large-scale users scenarios and extensive experiments demonstrate our scheme has high efficiency and high accuracy compared with non-private model.

Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yang Liu
2. Jiankun Hu
3. Yanming Zhu
4. Xuefei Yin
5. Qiang Yang
6. Tianjian Chen
7. Ming Ding
8. H. Vincent Poor

In Federated & Distributed Learning

1. Yang Liu
2. Qiang Yang
3. Tianjian Chen
4. Yan Kang
5. H. Vincent Poor
6. Yuxi Fan
7. Li Li
8. Kuo-Yi Lin

In Privacy & Security

1. Yang Liu
2. Yan Kang
3. Qiang Yang
4. Tianjian Chen
5. Yuxi Fan
6. Li Li
7. Kuo-Yi Lin
8. Chuan Ma

Citations in the corpus

• Federated Learning With Differential Privacy: Algorithms and Performance Analysis (2019, background): “However, FL also poses several key challenges, such as private information leakage, expensive communication costs between servers and clients, and device variability [9]–[14].”
• Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives (2021, methodology): “Furthermore, using federated machine learning, which helps in training algorithms over various decentralized edge-devices/servers holding local power consumption patterns without sharing them seems very promising for anomaly detection in energy consumption [256].”
• FedBCD: A Communication-Efficient Collaborative Learning Framework for Distributed Features (2022, background): “Hybrid approaches [22], [23] are also proposed to achieve better trade-off between accuracy and security.”
• A Survey on federated learning* (2020, background): “Scholars also designed secure aggregation protocol [9] [31] which is tolerant with arbitrary dropouts as long as surviving users are enough to join federate update.”
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey (2019, background): “Thus, the authors in [162] propose a hybrid solution which integrates both additively homomorphic encryption and DP in FL.”
• From distributed machine learning to federated learning: a survey (2021, methodology): “, the combination of DP and TEE in [58], the combination of encryption and DP [194], and the combination of DP and anti-GAN [166].”
• Efficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence (2020, background): “Some recent work [19], [22] can alleviate this problem by delegating the decryption operation”
• A Comprehensive Survey of Privacy-preserving Federated Learning (2021, methodology): “• In the FL, the intermediate weights or gradients communicated between participants and the aggregator may reveal sensitive information about the participants’ training datasets, as shown in References [96, 137, 198, 212].”
• A review of applications in federated learning (2020, background): “, (2019), Hard et al., (2019) and Yang, T. et al., (2018). Besides, emoji prediction is also a research hotspot (Ramaswamy et al.”