Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment

By Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, Zhenkai Liang. 2019

263 citations recorded by the source corpus.

The wide application of deep learning technique has raised new security concerns about the training data and test data. In this work, we investigate the model inversion problem under adversarial settings, where the adversary aims at inferring information about the target model's training data and test data from the model's prediction values. We develop a solution to train a second neural network that acts as the inverse of the target model to perform the inversion. The inversion model can be trained with black-box accesses to the target model. We propose two main techniques towards training the inversion model in the adversarial settings. First, we leverage the adversary's background knowledge to compose an auxiliary set to train the inversion model, which does not require access to the original training data. Second, we design a truncation-based technique to align the inversion model to enable effective inversion of the target model from partial predictions that the adversary obtains on victim user's data. We systematically evaluate our approach in various machine learning tasks and model architectures on multiple image datasets. We also confirm our results on Amazon Rekognition, a commercial prediction API that offers "machine learning as a service". We show that even with partial knowledge about the black-box model's training data, and with only partial prediction values, our inversion approach is still able to perform accurate inversion of the target model, and outperform previous approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Deduplicating Training Data Mitigates Privacy Risks in Language Models (2022, background): “Later model inversion attacks focus on fully recreating training samples given access to only a trained model (Hidano et al., 2017; Song & Raghunathan, 2020; Yang et al., 2019).”
NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, methodology): “2) Adversarial reconstruction: This threat model has also been considered with regards to adversarial reconstruction attack settings such as those considered in [28] based on activations obtained at the end of neural network.”
A Survey of Privacy Attacks in Machine Learning (2020, background): “Such an active attacker reached an attack accuracy of 92.1%.”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “The privacy attack could be defined as malicious attempts revealing training data, such as membership inference [109, 114, 125] and model inversion [34, 45, 146, 154, 161].”
Differential Privacy for Deep and Federated Learning: A Survey (2022, background): “Since, a malicious user (or the server) could reconstruct the users’ local dataset using only the local gradient sent to the server [10], [122], [123].”
Privacy Risks of General-Purpose Language Models (2020, background): “Later, some very recent works have proposed ﬁner-grained attacks which attempt to recover the exact training images or texts from the predictions [58], [77] or the gradients [47], [82] in an unknown mini-batch during the training phase.”
DarkneTZ: towards model privacy at the edge using trusted execution environments (2020, background): “DarkneTZ is not only capable of defending MIAs by controlling information from outputs, but also capable of defending other types of attacks such as training-based model inversion attack [16, 56] or GAN attack [21] as they also highly depend on the model’s outputs.”
Fawkes: Protecting Privacy against Unauthorized Deep Learning Models (2020, result): “Similarly, prior work [67] only retrieved 48,579 images.”