ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

By Ahmed Salem, Yang Zhang, Mathias Humbert, Mario Fritz, Michael Backes. 2018

1,184 citations recorded by the source corpus.

Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications. However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowledge of the target model structure, and having a dataset from the same distribution as the target model's training data. We relax all these key assumptions, thereby showing that such attacks are very broadly applicable at low cost and thereby pose a more severe risk than previously thought. We present the most comprehensive study so far on this emerging and developing threat using eight diverse datasets which show the viability of the proposed attacks across domains. In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, background): “Inference Parameter Inference Property Inference Access to Model Access to Model Output Membership Inference [13] # # # # Blackbox Logits Measuring Membership Privacy[35] # # # # Blackbox Logits ML-Leaks[36] # # # # Blackbox Logits The Natural Auditor [37] # # # # Blackbox Label LOGAN [38] # # # # B”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, methodology): “Attacks targeting data privacy include: the adversary inferring whether input examples were used to train the target model with membership inference attacks [38, 42, 59], learning global properties of training data with property inference attacks [12], reconstructing training data with model inversi”
Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning (2018, background): “n the literature. Privacy attacks. Shokri et al. [50] demonstrated a membership inference attack using shadow model training that learns if a victim’s data was used to train the model. Follow up work [45] showed that it is quite easy to launch this attack in a black-box setting. In addition, model i”
Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning (2018, background): “[17] extended the attacks of Shokri et al.”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “These attacks can be seen as privacy risk analysis tools [67], which help reveal how much the model has memorized the individual samples in its training set, and what the risk of individual users is [85, 69, 59, 81, 88].”
Information Leakage in Embedding Models (2020, background): “This subsequently led to a large body of work exploring this space [45, 48, 60, 61, 77].”
Evaluating Differentially Private Machine Learning in Practice (2019, positive): “[60] proposed more generic membership inference attacks by relaxing the requirements of Shokri et al.”
Membership Leakage in Label-Only Exposures (2020, methodology): “In contrast to the prior score-based attacks[25, 31, 35, 46, 48, 49, 57] that directly exploit confidence scores as analysis objects, we place our focus on the antithesis of this observation, i.”
Security and Privacy Challenges of Large Language Models: A Survey (2024, positive): “Salem et al. came up with the first effective defense mechanism against MIA [177].”
A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “The privacy attack could be defined as malicious attempts revealing training data, such as membership inference [109, 114, 125] and model inversion [34, 45, 146, 154, 161].”
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning (2018, background): “Attacks that compromise privacy of models in this setting include: model stealing [52,62,77], membership inference [66,67,71] which identiﬁes if a particular example was used during training, attribute inference [28,56] to identify properties that holds true for subsets of data, and model inversion…”
Zero-Shot Machine Unlearning (2022, background): “information leakage of the training data [25], [26].”
Label-Only Membership Inference Attacks (2020, methodology): “[14] show that attacks that rely on knowledge of query labels can often be matched by attacks that do not.”
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models (2019, background): “y breach would raise serious issues because training data either contains sensitive attributes (e.g., a patient’s disease history) or is cumbersome to collect. One such attack is membership inference [13,31,38,69,73] which aims to identify if a query data record was used to train a deep learning mod”
Membership Inference Attacks on Machine Learning: A Survey (2021, methodology): “[83] Salem et al.[174], Li and Zhang[112], Choquette et al.”
Survey of Attacks and Defenses on Edge-Deployed Neural Networks (2019, background): “Similarly, in [37], the authors propose two defenses:”
A Survey of Privacy Attacks in Machine Learning (2020, methodology): “Most often in the form of dropout, regularization, is proposed by multiple papers with varying levels of success [8, 30, 61, 79, 85, 88].”
BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2020, background): “For instance, multiple works have explored the security and privacy threats of data used to train DNNmodels, such as membership inference attack [30, 35, 36], dataset reconstruction attack [33], and property inference attack [8, 12].”