White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

By Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, Herv'e J'egou. 2019

462 citations recorded by the source corpus.

Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depend on the loss function, and thus black-box attacks are as good as white-box attacks. As the optimal strategy is not tractable, we provide approximations of it leading to several inference methods, and show that existing membership inference methods are coarser approximations of this optimal strategy. Our membership attacks outperform the state of the art in various settings, ranging from a simple logistic regression to more complex architectures and datasets, such as ResNet-101 and Imagenet.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, background): “Some studies (Sablayrolles et al., 2019; Truex et al., 2018; Yeom et al., 2018) attribute membership inference attacks to the generalization gap, the over-fitting of the model, and data memorization capabilities of neural networks.”
Counterfactual Memorization in Neural Language Models (2021, background): “Previous paper studies membership inference attack [Shokri et al., 2017, Sablayrolles et al., 2019, Long et al., 2020] where an attacker tries to figure out if a particular example exists in the training set.”
Measuring Forgetting of Memorized Training Examples (2022, background): “, 2017), current state-of-the-art MI attacks (Long et al., 2020; Sablayrolles et al., 2019; Watson et al., 2021; Carlini et al., 2021a) choose a carefully calibrated threshold for each example, which adjusts to the “hardness” of learning different examples.”
Information Leakage in Embedding Models (2020, background): “This subsequently led to a large body of work exploring this space [45, 48, 60, 61, 77].”
Security and Privacy Issues in Deep Learning (2018, background): “ship inference attack (MIA) [Shokri et al. 2017b; Song and Shmatikov 2018; Hayes et al. 2019] model determines whether the given data was used for the training step of the target model. Some studies [Sablayrolles et al. 2019; Truex et al. 2019] introduced the reason of feasibility of MIA as the over”
Radioactive data: tracing through training (2020, unknown): “Passive techniques to infer set membership (Shokri et al., 2017; Sablayrolles et al., 2019) require to know the trained architecture, as well as held-out data following the same distribution as that of the training set.”
Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations (2020, background): “Membership Inference Attacks [32,19,17,26,29,30,27] try to guess if a particular sample was used for training a model.”
On the Importance of Difficulty Calibration in Membership Inference Attacks (2021, methodology): “Formally, let Dshadow be a shadow dataset drawn from the same data distribution as the training set of h, and let A be a randomized training algorithm that samples from a distribution over models trained on Dshadow.”
Membership Inference Attacks on Machine Learning: A Survey (2021, background): “[133] 2019 S&P Classification White-box Non-zero gradient AP, AR Yelp-health CSI, Foursquare [147] 2019 S&P Classification White-box Intermediate computation Shadow training ASR CIFAR-100 Texas-100, Purchase-100 [186] 2019 S&P Classification Black-box Shadow training Prediction confidence ASR SVHN C”
Dataset Inference: Ownership Resolution in Machine Learning (2021, background): “MI attacks are shown to work in white-(Leino & Fredrikson, 2020; Sablayrolles et al., 2019) as well as black-box scenarios against various target models including generative models (Hayes et al., 2019).”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, background): “Additionally, new line of work demonstrated that it is possible to derive the bounds of success of a particular empirical class of attacks like membership inference from DP bounds (Yeom et al., 2017; Erlingsson et al., 2019b; Sablayrolles et al., 2019; Jayaraman et al., 2020), without having to do e”
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023, background): “models’ tendency to overfit their training data and therefore exhibit lower loss values for training members (Yeom et al., 2018; Sablayrolles et al., 2019).”
Membership Leakage in Label-Only Exposures (2020, background): “Various recent research [23, 25, 31, 35, 36, 45, 46, 48, 49, 54, 57, 60] has shown that ML models are vulnerable to privacy attacks.”
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models (2019, background): “For discriminative (classification) models, recent literature reports that the state-of-the-art black-box attack performs almost as well as the white-box attack [51, 57].”
PPFL: privacy-preserving federated learning with trusted execution environments (2021, methodology): “We follow a more practical approach based on the observation that membership-related information is only sensitive in the last DNN layer, making it vulnerable to MIAs as indicated in previous research [47, 49, 52, 59].”
Enhanced Membership Inference Attacks against Machine Learning Models (2021, background): “A substantial body of literature followed this work extending the shadow model attacks to different setting such as white box analysis [15, 23, 27], label-only access [5, 16], and federated learning [20, 23].”
Privacy Risks of Explaining Machine Learning Models (2019, background): “[31] have shown that an aacker can create an optimal black-box membership inference aack based only on the loss of the target model (under some assumptions).”
On the Privacy Risks of Model Explanations (2019, methodology): “Shadow model based attacks compare well to the optimal attack, with attacks based on three shadow model performing nearly at an optimal level; this is in line with results for loss-based attacks [34].”
Membership Inference Attacks From First Principles (2021, methodology): “[56] is the most direct influence for LiRA.”
Disparate Vulnerability: on the Unfairness of Privacy Attacks Against Machine Learning (2019, methodology): “We assume these Bayes adversaries have perfect knowledge of the underlying probability distributions and can thus use Bayes-optimal classi ers [8, 26]: A(y, ŷ) , arg max m∈{0,1} Pr[m | y, ŷ], A(y, ŷ, z) , arg max m∈{0,1} Pr[m | y, ŷ, z] .”