Model inversion attacks against collaborative inference

By Zecheng He, Tianwei Zhang, Ruby B. Lee. 2019

396 citations recorded by the source corpus.

The prevalence of deep learning has drawn attention to the privacy protection of sensitive data. Various privacy threats have been presented, where an adversary can steal model owners' private data. Meanwhile, countermeasures have also been introduced to achieve privacy-preserving deep learning. However, most studies only focused on data privacy during training, and ignored privacy during inference. In this paper, we devise a new set of attacks to compromise the inference data privacy in collaborative deep learning systems. Specifically, when a deep neural network and the corresponding inference task are split and distributed to different participants, one malicious participant can accurately recover an arbitrary input fed into this system, even if he has no access to other participants' data or computations, or to prediction APIs to query this system. We evaluate our attacks under different settings, models and datasets, to show their effectiveness and generalization. We also study the characteristics of deep learning models that make them susceptible to such inference privacy threats. This provides insights and guidelines to develop more privacy-preserving collaborative systems and algorithms.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, background): “[47] propose a new set of attacks to compromise the”
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion (2019, background): “Follow-up works have reﬁned and expanded the approach to new threat scenarios such as collaborative learning and membership detection [19, 64, 68].”
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey (2024, neutral): “Multiple types of inversion attacks [260], [261] have been proposed to reconstruct user’s data by hijacking the intermediate results.”
End-to-end privacy preserving deep learning on multi-institutional medical imaging (2021, neutral): “We note that while the data enjoys information-theoretic secrecy guarantees, the party requesting inference has access to the model’s predictions and can perform black-box membership inference 59 or model inversion attacks 60 .”
Looking Back on Learned Experiences For Class/task Incremental Learning (2022, methodology): “Inspired by (Fredrikson et al., 2015; He et al., 2019; Nayak et al., 2019), we propose a solution to reply the previous experiences by recovering the learner network memory.”
Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization (2021, methodology): “Model Inversion Attack (He et al., 2019; Salem et al., 2020) is able to recover the input data via an analysis of the model prediction.”
Vertical Federated Learning: Concepts, Advances, and Challenges (2022, methodology): “He et al. [95] proposed a black-box MI attack to learn features x B of the passive party B under the splitNN setting by training a shadow model ˆ G B to mimic the local model G B using auxiliary data.”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “The privacy attack could be defined as malicious attempts revealing training data, such as membership inference [109, 114, 125] and model inversion [34, 45, 146, 154, 161].”