Privacy Risks of Explaining Machine Learning Models

By Reza Shokri, Martin Strobel, Yair Zick. 2019

42 citations recorded by the source corpus.

Can we trust black-box machine learning with its decisions? Can we trust algorithms to train machine learning models on sensitive data? Transparency and privacy are two fundamental elements of trust for adopting machine learning. In this paper, we investigate the relation between interpretability and privacy. In particular we analyze if an adversary can exploit transparent machine learning to infer sensitive information about its training set. To this end, we perform membership inference as well as reconstruction attacks on two popular classes of algorithms for explaining machine learning models: feature-based and record-based influence measures. We empirically show that an attacker, that only observes the feature-based explanations, has the same power as the state of the art membership inference attacks on model predictions. We also demonstrate that record-based explanations can be effectively exploited to reconstruct significant parts of the training set. Finally, our results indicate that minorities and special cases are more vulnerable to these type of attacks than majority groups.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Explainable machine learning in deployment (2019, background): “Three organizations mentioned data privacy in the context of explainability, since in some cases explanations can be used to learn about the model [38, 63] or the training data [56].”
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects (2020, background): “Even under partial knowledge of the friend’s attributes, an adversary may use the information about the shared predictions in B Δ 𝒙 F to perform membership inference attacks [163] or infer missing attributes [59].”
LDP-Fed: federated learning with local differential privacy (2020, background): “This is a valuable feature as previous work [20, 24] has indicated that vulnerability to privacy attacks is not uniform and may be more acute for some participants’ datasets, leading to a desire for a stricter privacy guarantee.”
A Survey of Privacy Attacks in Machine Learning (2020, background): “We know, for instance, that adversarial training adversely affects membership inference [84] and that model censoring can still leak private attributes [87].”