Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
By Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, Hairong Qi. 2018
921 citations recorded by the source corpus.
Federated learning, i.e., a mobile edge computing framework for deep learning, is a recent advance in privacy-preserving machine learning, where the model is trained in a decentralized manner by the clients, i.e., data curators, preventing the server from directly accessing those private data from the clients. This learning mechanism significantly challenges the attack from the server side. Although the state-of-the-art attacking techniques that incorporated the advance of Generative adversarial networks (GANs) could construct class representatives of the global data distribution among all clients, it is still challenging to distinguishably attack a specific client (i.e., user-level privacy leakage), which is a stronger privacy threat to precisely recover the private data from a specific client. This paper gives the first attempt to explore user-level privacy leakage against the federated learning by the attack from a malicious server. We propose a framework incorporating GAN with a multi-task discriminator, which simultaneously discriminates category, reality, and client identity of input samples. The novel discrimination on client identity enables the generator to recover user specified private data. Unlike existing works that tend to interfere the training process of the federated learning, the proposed method works “invisibly” on the server side. The experimental results demonstrate the effectiveness of the proposed attacking approach and the superior to the state-of-the-art.
Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Qiang Yang
- Murali Annavaram
- Chaoyang He
- Yang Liu
- Bingsheng He
- Qinbin Li
- S. Avestimehr
- Abhishek Singh
In Federated & Distributed Learning
- Qiang Yang
- Murali Annavaram
- Chaoyang He
- Yang Liu
- Bingsheng He
- Qinbin Li
- S. Avestimehr
- Praneeth Vepakomma
In Privacy & Security
- Qiang Yang
- Chaoyang He
- Murali Annavaram
- Yan Kang
- Ramesh Raskar
- Praneeth Vepakomma
- Yang Liu
- Bingsheng He
Citations in the corpus
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “…challenge, existing research focuses on developing new adversarial attack and defense techniques to make federated learning robust [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 22], and proposing methods such as differential privacy (DP) and secure multiparty computation (SMPC) to protect…”
- Privacy preservation in federated learning: An insightful survey from the GDPR perspective (2020, background): “, active attack) [122].”
- Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment (2019, background): “Recent work also studies privacy issues in collaborative learning [48, 73] and ML-as-a-service platforms [67].”
- A Framework for Evaluating Gradient Leakage Attacks in Federated Learning (2020, background): “Privacy exploits at the prediction phase include model inversion, membership inference, and GAN-based reconstruction attack [13,14,38].”
- Trustworthy artificial intelligence (2020, background): “…despite increased confidentiality, research has also shown that federated learning is potentially vulnerable to inference attacks, whereby an adversary can aim to extract information about private training data by inferring the AI model multiple times (Melis et al. 2019; Wang et al. 2019).”
- A Comprehensive Survey of Privacy-preserving Federated Learning (2021, methodology): “• The inference of class representatives aims to generate representative samples, which are not real data instances of training datasets but can be used to study sensitive information about the training datasets [53, 75, 192].”
- Inverting Gradients - How easy is it to break privacy in federated learning? (2020, methodology): “Recent works on privacy attacks in federated learning setups ([25, 24, 31, 35, 34]) have hinted at the fact that previous hopes that “Privacy is enhanced by the ephemeral and focused nature of the [Federated Learning] updates” [3] are not true in general.”
- Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, background): “In [27], the authors study the privacy risk of FL, and propose a GAN-based attack that allows servers to target a specific client and compromise the client level privacy.”
- A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, background): “[100] are the first to consider attacks in FL from a malicious server rather than clients.”
- Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices (2019, background): “Wang et al. [35] proposed a framework incorporating generative adversarial network with a multitask discriminator to solve the user-level privacy leakage in FL against attacks from a malicious server.”
- Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems (2021, background): “Recent research on federated learning has focused on reducing communication overhead [9], [10], [26], [27] and protecting privacy [28], [29], [30], [31], but only a few studies have noticed the problem of accuracy degradation due to imbalance [15], [32], [33].”
- Federated Learning in Smart City Sensing: Challenges and Opportunities (2020, neutral): “The study in [180] showed how Generative Adversarial networks (GANs) with a multitask discriminator can extract user specific data quietly on the server side.”