Machine Learning with Membership Privacy using Adversarial Regularization

By Milad Nasr, Reza Shokri, Amir Houmansadr. 2018

554 citations recorded by the source corpus.

Machine learning models leak significant amount of information about their training sets, through their predictions. This is a serious privacy concern for the users of machine learning as a service. To address this concern, in this paper, we focus on mitigating the risks of black-box inference attacks against machine learning models. We introduce a mechanism to train models with membership privacy, which ensures indistinguishability between the predictions of a model on its training data and other data points (from the same distribution). This requires minimizing the accuracy of the best black-box membership inference attack against the model. We formalize this as a min-max game, and design an adversarial training algorithm that minimizes the prediction loss of the model as well as the maximum gain of the inference attacks. This strategy, which can guarantee membership privacy (as prediction indistinguishability), acts also as a strong regularizer and helps generalizing the model. We evaluate the practical feasibility of our privacy mechanism on training deep neural networks using benchmark datasets. We show that the min-max strategy can mitigate the risks of membership inference attacks (near random guess), and can achieve this with a negligible drop in the model's prediction accuracy (less than 4%).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Extracting Training Data from Large Language Models (2020, background): “In their most basic form, past membership inference attacks rely on the observation that models tend to assign higher confidence to examples that are present in the training data [41].”
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks (2018, background): “Other related work [38] studies how to apply adversarial regularization to reduce the risk of black-box membership inference attacks, although using different approach than taken by prior work.”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “[33] propose to measure the performance of membership inference attack at each training step and use the measurement as a new regularizer.”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “[39] design a privacy mechanism to adversarially train centralized machine learning models with provable protections against membership inference.”
Comprehensive Privacy Analysis of Deep Learning: Stand-alone and Federated Learning under Passive and Active White-box Inference Attacks (2018, background): “Several works [36]–[38] showed that by using adversarial training, one can ﬁnd a better trade-off between privacy and model accuracy.”
Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, neutral): “…a privacy-preserving data ﬁlter in a multi-party setting [15], learning a privacy-preserving record representation [10], while, in parallel to our work, Nasr et al. [31] use it to protect against membership privacy attacks [40], i.e., hiding whether a record was part of a training dataset or not.”
Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning (2018, background): “Several works [28], [29], [30] have shown that by using adversarial training, one can find a better trade-off between privacy and model accuracy.”
LOGAN: Membership Inference Attacks Against Generative Models (2017, background): “[43] leverage adversarial regularizers to design a privacy-preserving training mechanism with provable protections against membership inference attacks against discriminative models.”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “Advances in privacy-preserving machine learning [27, 81, 118] present the possibility that the queer community might benefit from AI systems while minimising the risk of information leakage.”
Measuring Forgetting of Memorized Training Examples (2022, background): “Resulting privacy concerns have led to a variety of techniques for private machine learning, including differentially private training [1, 61, 62, 23, 53], machine unlearning [5, 58, 67], and various heuristics like regularization [56], data augmentation [2] or gradient clipping [10, 72, 32].”
On the Privacy Risks of Model Explanations (2019, methodology): “[29] M Nasr, R Shokri, and A Houmansadr.”
Privacy Risks of Explaining Machine Learning Models (2019, methodology): “[26] use adversarial regularization, while Papernot et al.”
A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “• The inference of memberships aims to determine whether a data sample has been used for model training [42, 114, 123, 129, 130, 139, 160].”
Membership Inference Attacks From First Principles (2021, background): “As a result it will be necessary to re-investigate prior results from this perspective: • Do previously-“broken” [26, 45] defenses prevent our attack? Prior defenses were only ever shown to be ineffective at preventing an adversary from succeeding on average—not confidently at low false-positive rat”
The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets (2018, result): “ the exposure of secrets. Other related work [41] studies membership attacks on differentially private training, although in the setting of a distributed honest-but-curious server. Other related work [38] studies how to apply adversarial regularization to reduce the risk of black-box membership infe”
Membership Inference Attacks on Machine Learning: A Survey (2021, methodology): “The adversarial regularization [146] and Mixup + MMD [110] are specially designed regularization techniques that aim to mitigate MIAs.”
Towards Demystifying Membership Inference Attacks (2018, background): “y, to gain this level of protection the noise introduced to the model decreases model accuracy to 35%. This is a signiﬁcant challenge in the mitigation of membership inference attacks. Recent work in [44] has proposed a regularization technique speciﬁcally targeting membership inference attacks. Whi”
Security and Privacy Challenges of Large Language Models: A Survey (2024, unknown): “In order to mitigate MIA in the language domain, several mechanisms are proposed, including dropout, model stacking, differential privacy [173], and adversarial regularization [176].”
Zero-Shot Machine Unlearning (2022, background): “erable defense methods have been proposed based on reducing the model overfitting [32], perturbation of the posteriors [33], and adversarial training [34].”