Towards Measuring Membership Privacy

By Yunhui Long, Vincent Bindschaedler, Carl A. Gunter. 2017

89 citations recorded by the source corpus.

Machine learning models are increasingly made available to the masses through public query interfaces. Recent academic work has demonstrated that malicious users who can query such models are able to infer sensitive information about records within the training data. Differential privacy can thwart such attacks, but not all models can be readily trained to achieve this guarantee or to achieve it with acceptable utility loss. As a result, if a model is trained without differential privacy guarantee, little is known or can be said about the privacy risk of releasing it. In this work, we investigate and analyze membership attacks to understand why and how they succeed. Based on this understanding, we propose Differential Training Privacy (DTP), an empirical metric to estimate the privacy risk of publishing a classier when methods such as differential privacy cannot be applied. DTP is a measure of a classier with respect to its training dataset, and we show that calculating DTP is efficient in many practical cases. We empirically validate DTP using state-of-the-art machine learning models such as neural networks trained on real-world datasets. Our results show that DTP is highly predictive of the success of membership attacks and therefore reducing DTP also reduces the privacy risk. We advocate for DTP to be used as part of the decision-making process when considering publishing a classifier. To this end, we also suggest adopting the DTP-1 hypothesis: if a classifier has a DTP value above 1, it should not be published.

Subfields: Artificial Intelligence, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks (2018, background): “More closely related to our paper is work which produces measurements for how likely it is that membership inference attacks will be possible [30] by developing the Differential Training Privacy metric for cases when differentially private training will not be possible.”
Privacy in Deep Learning: A Survey (2020, background): “Inference Parameter Inference Property Inference Access to Model Access to Model Output Membership Inference [13] # # # # Blackbox Logits Measuring Membership Privacy[35] # # # # Blackbox Logits ML-Leaks[36] # # # # Blackbox Logits The Natural Auditor [37] # # # # Blackbox Label LOGAN [38] # # # # B”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “The performance of membership inference attack is related to the target model’s sensitivity with regard to training data [29].”
Understanding Membership Inferences on Well-Generalized Learning Models (2018, background): “The risk of membership inferences can be empirically measured based on the influence of each training record [28].”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “These attacks can be seen as privacy risk analysis tools [67], which help reveal how much the model has memorized the individual samples in its training set, and what the risk of individual users is [85, 69, 59, 81, 88].”
Demystifying Membership Inference Attacks in Machine Learning as a Service (2019, unknown): “In [7], the authors claim that the shadow model implementation of a membership inference attack requires the shadow models be trained in a similar way to the target model, an assumption followed by later work such as [42].”
Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning (2019, background): “More recently, multiple membership inference attacks have been proposed with new attacking techniques or targeting on different types of ML models [19, 27, 28, 31, 32, 38, 42, 53].”
On the Privacy Risks of Model Explanations (2019, background): “Several works have exploited this fact to define simple loss-based attacks [29, 39, 54].”
Label-Only Membership Inference Attacks (2020, background): “confidence vector h(x), y train, data, label [7] confidence vector h(x) train, data [38] confidence vector h(x) – [14] confidence vector h(x) – [14] confidence vector L (h (x) , y) – [11] label-only argmaxh(x), y label [11]”
When Relaxations Go Bad: "Differentially-Private" Machine Learning (2019, unknown): “A more targeted approach was proposed by Long et al. [44] where the shadow models are trained with and without a targeted input record t .”
ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models (2018, background): “Following [36], many recent works have studied membership inference against machine learning from different angles [26], [27], [45].”
Dynamic Backdoor Attacks Against Machine Learning Models (2020, background): “Beside the above, there are multiple other types of attacks against machine learning models, such as membership inference [39], [16], [13], [34], [35], [24], [14], [25], [50],”
Membership Inference Attacks From First Principles (2021, background): “[35] has a much lower average accuracy (53.”
Towards Demystifying Membership Inference Attacks (2018, background): “trained in a similar way to the target model, an assumption followed by later work such as [27].”
MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples (2019, result): “While most of the previous works concentrated on classification models [33, 34, 42, 43, 56, 58, 69], Hayes et al. [21] studied membership inference against generative models, in particular generative adversarial networks (GANs) [18].”
Membership Leakage in Label-Only Exposures (2020, background): “In contrast to the prior score-based attacks[25, 31, 35, 46, 48, 49, 57] that directly exploit confidence scores as analysis objects, we place our focus on the antithesis of this observation, i.”
Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference (2019, background): “ of collaborative, or federated, learning. Black-box attacks. We study membership inference as it applies to deep networks in classic supervised learning problems. Most of the prior work in this area [27, 28, 35, 38, 47] has used the black-box threat model.Yeom et al.[47] showed that generalization ”