On the Importance of Difficulty Calibration in Membership Inference Attacks

By Lauren Watson, Chuan Guo, Graham Cormode, Alexandre Sablayrolles. 2021

197 citations recorded by the source corpus.

The vulnerability of machine learning models to membership inference attacks has received much attention in recent years. However, existing attacks mostly remain impractical due to having high false positive rates, where non-member samples are often erroneously predicted as members. This type of error makes the predicted membership signal unreliable, especially since most samples are non-members in real world applications. In this work, we argue that membership inference attacks can benefit drastically from \emph{difficulty calibration}, where an attack's predicted membership score is adjusted to the difficulty of correctly classifying the target sample. We show that difficulty calibration can significantly reduce the false positive rate of a variety of existing attacks without a loss in accuracy.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Deduplicating Training Data Mitigates Privacy Risks in Language Models (2022, neutral): “Similar ﬁndings have been noted in Watson et al. (2021) and Carlini et al. (2021a).”
Measuring Forgetting of Memorized Training Examples (2022, background): “While early techniques applied the same threshold to all examples [81, 69], current state-of-the-art membership inference attacks [52, 65, 79, 9] choose a carefully calibrated threshold for each example.”
Federated Learning with Buffered Asynchronous Aggregation (2021, background): “(2018)) provide protection against inference attacks (Watson et al., 2021; Carlini et al., 2020).”
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023, methodology): “For reference-based attacks, we follow recent implementations (Mireshghallah et al., 2022a,b; Watson et al., 2022) and use reference data to train a single reference model of the same architecture as the target model.”
Detecting Pretraining Data from Large Language Models (2023, methodology): “MIA methods for finetuning (Carlini et al., 2022; Watson et al., 2022) usually calibrate the target model probabilities of an example using a shadow reference model that is trained on a similar data distribution.”
Enhanced Membership Inference Attacks against Machine Learning Models (2021, background): “Partially motivated by these observations, and by prior works [22] that computes likelihood ratio to launch powerful membership inference attacks, recent works [2, 36] design per-example attacks that perform better with higher confidence (e.”
Membership Inference Attacks From First Principles (2021, methodology): “The simplest method to evaluate attack efficacy is through a standard “accuracy” metric that measures how often an attack correctly predicts membership on a balanced dataset of members and non-members [6, 18, 33, 46, 56, 60, 61, 66, 68, 70].”