Privacy Risks of Securing Machine Learning Models against Adversarial Examples

By Liwei Song, Reza Shokri, Prateek Mittal. 2019

304 citations recorded by the source corpus.

The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is thus unclear whether the defense methods in one domain will have any unexpected impact on the other domain. In this paper, we take a step towards resolving this limitation by combining the two domains. In particular, we measure the success of membership inference attacks against six state-of-the-art defense methods that mitigate the risk of adversarial examples (i.e., evasion attacks). Membership inference attacks determine whether or not an individual data record has been part of a model's training set. The accuracy of such attacks reflects the information leakage of training algorithms about individual members of the training set. Adversarial defense methods against adversarial examples influence the model's decision boundaries such that model predictions remain unchanged for a small area around each input. However, this objective is optimized on training data. Thus, individual data records in the training set have a significant influence on robust models. This makes the models more vulnerable to inference attacks. To perform the membership inference attacks, we leverage the existing inference methods that exploit model predictions. We also propose two new inference methods that exploit structural properties of robust models on adversarially perturbed data. Our experimental evaluation demonstrates that compared with the natural training (undefended) approach, adversarial defense methods can indeed increase the target model's risk against membership inference attacks. When using adversarial defenses to train the robust models, the membership inference advantage increases by up to 4.5 times compared to the naturally undefended models. Beyond revealing the privacy risks of adversarial defenses, we further investigate the factors, such as model capacity, that influence the membership information leakage.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Privacy & Security

Citations in the corpus

Privacy Risks of General-Purpose Language Models (2020, unknown): “As a complement to model inversion attack, Shokri et al. devised the ﬁrst membership inference attack against machine learning models [63], which aroused wide research interests [50], [59], [66], [78] in the past few years.”
Improving Robustness using Generated Data (2021, background): “We note that while robustness and invariance to input perturbations is crucial to the deployment of machine learning models in various applications, it can also have broader negative impacts to society such as hindering privacy [63] or increasing bias [68].”
A Survey of Privacy Attacks in Machine Learning (2020, background): “In the active attack scenario, the attacker, which is also a local participant, alters the gradient updates to perform a gradient ascent instead of descent for the data whose membership is under question.”
Data Augmentation Can Improve Robustness (2021, background): “…training points on the model when using adversarial training can reduce the sensitivity of the model and lead to larger biases [62], (3) Song et al. [46] show that improved robustness increases the success of privacy attacks and that robust models are more sensitive to membership inference attacks.”
Membership Inference Attacks on Machine Learning: A Survey (2021, unknown): “MNIST [103] 70,000 28 × 28 × 1 10 [22, 23, 25–27, 30, 66, 72, 78, 107, 110, 112, 118, 123, 124, 143, 148, 161, 162, 164, 165, 174, 181, 198–200, 205, 208, 227, 237] Fashion-MNIST [219] 70,000 28 × 28 × 1 10 [41, 66, 78, 80, 93, 94, 143, 161, 165, 187]”
Systematic Evaluation of Privacy Risks of Machine Learning Models (2020, methodology): “[44] find that the metric of prediction confidence of correct label F(x)y can be compared with a certain threshold value to achieve similar attack performance as NN-based attacks.”
RobustBench: a standardized adversarial robustness benchmark (2020, background): “Recent 319 works have highlighted that robust training exacerbates this problem [119].”
Enhanced Membership Inference Attacks against Machine Learning Models (2021, background): “Beyond these theoretical analyses, membership inference attacks are also used to empirically study the trade-offs between privacy and other desirable characteristics for machine learning models such as fairness [Chang and Shokri, 2020], robustness to adversarial examples [Song et al., 2019], and mod”
Membership Leakage in Label-Only Exposures (2020, background): “In contrast to the prior score-based attacks[25, 31, 35, 46, 48, 49, 57] that directly exploit confidence scores as analysis objects, we place our focus on the antithesis of this observation, i.”
Label-Only Membership Inference Attacks (2020, background): “[45] also make use of adversarial examples to infer membership.”
MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples (2019, background): “Multiple studies have shown that such black-box ML classifier is vulnerable to membership inference attacks [43, 56, 58, 59].”
On the Privacy Risks of Model Explanations (2019, background): “However, robustness results in vulnerability to membership inference [43].”