Demystifying Membership Inference Attacks in Machine Learning as a Service

By Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, Wenqi Wei. 2019

330 citations recorded by the source corpus.

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and corresponding attack models are largely transferable. Though different model types display different vulnerabilities to membership inference, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning exposes vulnerabilities to membership inference risks when the adversary is a participant. We also discuss countermeasure and mitigation strategies.

Subfields: Artificial Intelligence, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks (2019, background): “Membership attacks [28, 15, 24, 14, 26] attempt to determine whether a particular cohort of data was used for training, without any constructive indication on how to actively forget it.”
A Survey of Privacy Attacks in Machine Learning (2020, methodology): “, an attack to one target model transfers to another target if the training dataset was the same [92].”
Membership Inference Attacks on Machine Learning: A Survey (2021, methodology): “[105] experimentally evaluate membership inference on NN model, logistic regression model, Naive Bayes model, k-nearest neighbor model, and decision tree model on seven datasets and the results shows that the decision tree model has the highest attack precision for six datasets and Naive Bayes model”
Machine Learning for Synthetic Data Generation: a Review (2023, background): “For deep neural network based approaches, membership inference attack [155], [156] would identify if an input is in the training data or not and thus can be used to determine how close the synthesized data is to the original data.”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “The privacy attack could be defined as malicious attempts revealing training data, such as membership inference [109, 114, 125] and model inversion [34, 45, 146, 154, 161].”
LDP-Fed: federated learning with local differential privacy (2020, background): “ient in protecting the privacy of the underlying training data from known inference attacks [16]. Model parameters exchanged during the training process [16] as well as outputs from the trained model [21, 25] remain as attack surfaces for privacy leakage. Existing solutions to protect FL systems fro”
Eavesdrop the Composition Proportion of Training Labels in Federated Learning (2019, background): “The former one assumes that the attacker compromises some participated devices and uses them to achieve malicious intentions, e.g. importing backdoor to FL [14, 15], adversarial poisoning [16, 17, 18], membership or property inference [19, 20, 21, 22], and reconstruction attack [23, 24, 25].”
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, methodology): “To address this issue, a grey-box attacker was introduced in [33], which is a black-box attacker with some specific statistical knowledge not publicly available that concerns her victim.”
Security and Privacy Challenges of Large Language Models: A Survey (2024, neutral): “Protecting LLMs from security and privacy attacks is imperative to preserve the reliability and integrity of this complex AI system [143].”
A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “• Studies [75, 160, 228] on inference attacks have shown that sensitive information can be extracted from the final model or an inference API if the final model or the query results are not protected properly.”
Data Poisoning Attacks Against Federated Learning Systems (2020, background): “he rising popularity of FL has led to the investigation of dierent attacks in the context of FL, such as backdoor attacks [2,46], gradient leakage attacks [18,27,59] and membership inference attacks [31,47,48]. Most closely related to our work are poisoning attacks in FL. There are two types of pois”
Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations (2020, methodology): “e extend this framework to activations, and show that using an NTK based scrubbing procedure uniformly improves the scrubbing procedure in all metrics that they consider. Membership Inference Attacks [32,19,17,26,29,30,27] try to guess if a particular sample was used for training a model. Since a mo”