Towards Demystifying Membership Inference Attacks

By Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, Wenqi Wei. 2018

120 citations recorded by the source corpus.

Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. Aiming at providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and its attack models are largely transferable. Though different model types display different vulnerabilities to membership inferences, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning in federated systems exposes vulnerabilities to membership inference risks when the adversary is a participant in the federation. We also discuss countermeasure and mitigation strategies.

Subfields: Artificial Intelligence, Computer Networks and Communications, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Safety, Robustness & Interpretability

Citations in the corpus

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks (2018, background): “Membership inference attacks have seen further study, including examining why membership inference is possible [50], or mounting inference attacks on other forms of generative models [23].”
Privacy in Deep Learning: A Survey (2020, background): “Some studies [44, 45, 15] attribute membership inference attacks to the generalization gap, the over-fitting of the model, and data memorization capabilities of neural networks.”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, methodology): “ was used to train the model or not. Membership inference attacks against aggregate statistics are well-known [14,27,50], and recent work demonstrated black-box membership inference against ML models [24,34,53,58], as discussed in Section10. The ability of an adversary to infer the presence of a spe”
Auditing Data Provenance in Text-Generation Models (2018, background): “istics in general [8]. Shokri et al. [28] develop black-box membership inference techniques against ML models which perform best when the target model is overfitted to the training data. Truex et al. [32] extend and generalize this work to white-box and federated-learning settings. Rahman et al. [27”
LOGAN: Membership Inference Attacks Against Generative Models (2017, methodology): “es of faces. By contrast, our evaluation is performed on 13,233, 60,000, and 88,702 images for the LFW, CIFAR-10, and Diabetic Retinopathy datasets, respectively (see Section5). Finally, Truex et al. [63] show how membership inference attacks are data-driven and largely transferable, while Melis et ”
Does learning require memorization? a short tale about a long tail (2019, background): “…context of user data privacy, memorization is known to lead to ability to mount black-box membership inference attacks (that discover the presence of a speciﬁc data point in the dataset) [SSSS17; LBG17; Lon+18; TLGYW18] as well as ability to extract planted secrets from language models [CLKES19].”
Membership Leakage in Label-Only Exposures (2020, methodology): “Various recent research [23, 25, 31, 35, 36, 45, 46, 48, 49, 54, 57, 60] has shown that ML models are vulnerable to privacy attacks.”
Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System? (2019, background): “Truex et al. (2018) presents a comprehensive evaluation of different model types, training data, and attack strategies; Borrowing ideas from adversarial learning and minimax games, Hayes et al. (2017) proposes attack methods based on generative adversarial networks, while Nasr et al. (2018)…”
The Natural Auditor: How To Tell If Someone Used Your Words To Train Their Model (2018, background): “Truex et al. [37] extend and generalize this work, including white-box and federated-learning setting.”
Do Language Models Plagiarize? (2022, background): “ing results, a growing body of literature gave rise to privacy violations of a neural language model resulting from data memorization (Carlini et al., 2019; Thakkar et al., 2021; Truex et al., 2018; Arpit et al., 2017; Meehan et al., 2020).”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “In this section, we examine the range of existing defense methods against various attacks and vulnerabilities associated with LLMs 1 .”
The Limitations of Federated Learning in Sybil Settings (2020, background): “Model free riding [34] Membership inference [41,42,54]”
Data Poisoning Attacks Against Federated Learning Systems (2020, background): “The rising popularity of FL has led to the investigation of diﬀerent attacks in the context of FL, such as backdoor attacks [2,46], gradient leakage attacks [18,27,59] and membership inference attacks [31,47,48].”
Systematic Evaluation of Privacy Risks of Machine Learning Models (2020, methodology): “In this paper, we critically examine how previous work [19,29,30,36,39,44] has evaluated the membership inference privacy risks of machine learning models, and demonstrate two key limitations that lead to a severe underestimation of privacy risks.”
A Simple and Efficient Federated Recommender System (2019, background): “Nevertheless, several research works show that FL, under certain circumstances, may lead to information leakage about users’ training data [14, 26].”
A Framework for Evaluating Gradient Leakage Attacks in Federated Learning (2020, methodology): “This set of experiments compares client privacy leakage attacks with two existing training data inference attacks: Melis et al [29] is based on membership-attack [34,35] and can make partial property inference on a carefully selected subset of data.”
Inference Attacks Against Collaborative Learning (2018, methodology): “ was used to train the model or not. Membership inference attacks against aggregate statistics are well-known [14,27,50], and recent work demonstrated black-box membership inference against ML models [24,34,53,58], as discussed in Section10. The ability of an adversary to infer the presence of a spe”
Membership Inference Attacks From First Principles (2021, methodology): “The simplest method to evaluate attack efficacy is through a standard “accuracy” metric that measures how often an attack correctly predicts membership on a balanced dataset of members and non-members [6, 18, 33, 46, 56, 60, 61, 66, 68, 70].”
Biscotti: A Blockchain System for Private and Secure Federated Learning (2021, background): “By observing a peer’s model updates from each verification round, an adversary can perform an information leakage attack [13], [20], [51], [52], [53], [54] and recover details about a victim’s training data.”