The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks

By Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, Dawn Song. Computer Vision and Pattern Recognition, 2019

533 citations recorded by the source corpus.

This paper studies model-inversion attacks, in which the access to a model is abused to infer information about the training data. Since its first introduction by~\cite{fredrikson2014privacy}, such attacks have raised serious concerns given that training data usually contain privacy sensitive information. Thus far, successful model-inversion attacks have only been demonstrated on simple models, such as linear regression and logistic regression. Previous attempts to invert neural networks, even the ones with simple architectures, have failed to produce convincing results. Here we present a novel attack method, termed the \emph{generative model-inversion attack}, which can invert deep neural networks with high success rates. Rather than reconstructing private training data from scratch, we leverage partial public information, which can be very generic, to learn a distributional prior via generative adversarial networks (GANs) and use it to guide the inversion process. Moreover, we theoretically prove that a model's predictive power and its vulnerability to inversion attacks are indeed two sides of the same coin---highly predictive models are able to establish a strong correlation between features and labels, which coincides exactly with what an adversary exploits to mount the attacks. Our extensive experiments demonstrate that the proposed attack improves identification accuracy over the existing work by about $75\%$ for reconstructing face images from a state-of-the-art face recognition classifier. We also show that differential privacy, in its canonical form, is of little avail to defend against our attacks.

Subfields: Artificial Intelligence, Ocean Engineering

View Paper: https://www.semanticscholar.org/paper/babf3916f3734fceeccf7aecb8da2353d92451e8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Ocean Engineering

Citations in the corpus

A Survey of Privacy Attacks in Machine Learning (2020, methodology): “Such an active attacker reached an attack accuracy of 92.1%.”
Inverting Gradients - How easy is it to break privacy in federated learning? (2020, background): “Model inversion generally is challenging for deeper neural network architectures [33] if no additional information is given [9, 33].”
Towards Unbounded Machine Unlearning (2023, background): “…of a subset of the training set from the weights of a trained model is hard, since deep models memorize information about speciﬁc instances [Zhang et al., 2020, 2021, Arpit et al., 2017] and their highly non-convex nature makes it difﬁcult to trace the effect of each example on the model’s…”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “The privacy attack could be defined as malicious attempts revealing training data, such as membership inference [109, 114, 125] and model inversion [34, 45, 146, 154, 161].”
Extracting Training Data from Diffusion Models (2023, background): “We instead provide guidance by performing a form of attribute inference attack [38, 80, 81] that we call an “inpainting attack”.”
Approximate Data Deletion from Machine Learning Models: Algorithms and Evaluations (2020, background): “For example, recent works show how one can reconstruct training data by attacking vision and NLP models (Zhang et al., 2019).”
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy (2022, background): “In Zhang et al. (2020), model inversion success is measured based on pixel similarity and feature space similarity to training images.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[155] proposed a generative modelinversion attack that trains a GAN to estimate the distributional prior of the target model’s training data.”
Membership Leakage in Label-Only Exposures (2020, background): “Various recent research [23, 25, 31, 35, 36, 45, 46, 48, 49, 54, 57, 60] has shown that ML models are vulnerable to privacy attacks.”
Bounding Training Data Reconstruction in Private (Deep) Learning (2022, background): “Membership inference is empirically much easier than powerful attacks such as training data reconstruction (Carlini et al., 2019; Zhang et al., 2020; Balle et al., 2022), and hence it may be possible to provide a strong semantic guarantee against data reconstruction attacks even when membership…”
CLeaR: A Unified Framework for Resolving the Leakage-Degradation Dilemma in Style Transfer (2026, unknown): “Early MI approaches primarily target classification networks, using gradient-based optimization and generative priors to reconstruct the original inputs from deep representations [50, 13, 36].”