MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples

By Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, Neil Zhenqiang Gong. Conference on Computer and Communications Security, 2019

475 citations recorded by the source corpus.

In a membership inference attack, an attacker aims to infer whether a data sample is in a target classifier's training dataset or not. Specifically, given a black-box access to the target classifier, the attacker trains a binary classifier, which takes a data sample's confidence score vector predicted by the target classifier as an input and predicts the data sample to be a member or non-member of the target classifier's training dataset. Membership inference attacks pose severe privacy and security threats to the training dataset. Most existing defenses leverage differential privacy when training the target classifier or regularize the training process of the target classifier. These defenses suffer from two key limitations: 1) they do not have formal utility-loss guarantees of the confidence score vectors, and 2) they achieve suboptimal privacy-utility tradeoffs. In this work, we propose MemGuard,the first defense with formal utility-loss guarantees against black-box membership inference attacks. Instead of tampering the training process of the target classifier, MemGuard adds noise to each confidence score vector predicted by the target classifier. Our key observation is that attacker uses a classifier to predict member or non-member and classifier is vulnerable to adversarial examples.Based on the observation, we propose to add a carefully crafted noise vector to a confidence score vector to turn it into an adversarial example that misleads the attacker's classifier. Specifically, MemGuard works in two phases. In Phase I, MemGuard finds a carefully crafted noise vector that can turn a confidence score vector into an adversarial example, which is likely to mislead the attacker's classifier to make a random guessing at member or non-member. We find such carefully crafted noise vector via a new method that we design to incorporate the unique utility-loss constraints on the noise vector. In Phase II, MemGuard adds the noise vector to the confidence score vector with a certain probability, which is selected to satisfy a given utility-loss budget on the confidence score vector. Our experimental results on three datasets show that MemGuard can effectively defend against membership inference attacks and achieve better privacy-utility tradeoffs than existing defenses. Our work is the first one to show that adversarial examples can be used as defensive mechanisms to defend against membership inference attacks.

Subfields: Artificial Intelligence, Safety Research

View Paper: https://www.semanticscholar.org/paper/857986936ac87e1fa7c0a9b2923908f6d0656fd2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Systematic Evaluation of Privacy Risks of Machine Learning Models (2020, methodology): “MemGuard [20] Location30 50.”
Label-Only Membership Inference Attacks (2020, methodology): “In contrast, the second category of adversary-aware defenses explicitly aim to minimize the membership inference leakage as computed by a particular attack [15, 16, 17].”
Security and Privacy Challenges of Large Language Models: A Survey (2024, unknown): “In gradient-based attacks, the adversary needs the white-box information of a model, which is sometimes impractical.”
Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning (2019, unknown): “Besides the above, there exist a wide range of other attacks and defenses on machine learning models [4,5,8,9,13,14,16, 17, 22, 23, 25, 26, 35, 41, 43, 44, 46, 48–50, 52, 54–56]”
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023, methodology): “Beyond approaches such as confidence score perturbation (Jia et al., 2019) and specific regularization techniques (Mireshghallah et al.”
Membership Leakage in Label-Only Exposures (2020, methodology): “We adopt the original implementation of MemGuard,6 and an open-source version of the adversarial regularization.7 For each mechanism, we train 3 target models (CIFAR-10,M−2) using different hyper-parameters.”
A Survey of Privacy Attacks in Machine Learning (2020, unknown): “Adding noise calculated using adversarial learning, also thwarts membership inference attacks [42].”
Membership Inference Attacks on Machine Learning: A Survey (2021, methodology): “Image Data Classification Colored-MNIST [3] 70,000 28×28×1 2 [196] CH-MNIST [92] 5,000 150×150×1 8 [76, 86, 161] SVHN [149] 99,289 32×32×3 10 [85, 165, 186] Yale Face [106] 2,414 168×192×1 38 [187] RCV1X [108] 800,000 N.”
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models (2019, background): “While numerous literature is dedicated to MIA against discriminative models [33, 45, 48, 50, 58, 60, 68], the attack on generative models has not received equal attention, despite its practical importance.”
Dynamic Backdoor Attacks Against Machine Learning Models (2020, background): “Other works explore the adversarial example’s potentials in preserving the user’s privacy in multiple domains [30], [18], [51], [19].”
DarkneTZ: towards model privacy at the edge using trusted execution environments (2020, methodology): “In recent work, carefully crafted noise is added to confidence scores by applying adversarial examples [29].”
Zero-Shot Machine Unlearning (2022, background): “erable defense methods have been proposed based on reducing the model overfitting [32], perturbation of the posteriors [33], and adversarial training [34].”
On the Importance of Difficulty Calibration in Membership Inference Attacks (2021, background): “To counter these attacks, prior work considered obfuscating the model’s output by returning only the top label or modifying the predicted values (Shokri et al., 2017; Jia et al., 2019).”