Selective Differential Privacy for Language Modeling

By Weiyan Shi, Aiqiang Cui, Evan Li, Ruoxi Jia, Zhou Yu. 2021

91 citations recorded by the source corpus.

With the increasing applications of language models, it has become crucial to protect these models from leaking private information. Previous work has attempted to tackle this challenge by training RNN-based language models with differential privacy guarantees.However, applying classical differential privacy to language models leads to poor model performance as the underlying privacy notion is over-pessimistic and provides undifferentiated protection for all tokens in the data. Given that the private information in natural language is sparse (for example, the bulk of an email might not carry personally identifiable information), we propose a new privacy notion, selective differential privacy, to provide rigorous privacy guarantees on the sensitive portion of the data to improve model utility. To realize such a new notion, we develop a corresponding privacy mechanism, Selective-DPSGD, for RNN-based language models. Besides language modeling, we also apply the method to a more concrete application – dialog systems. Experiments on both language modeling and dialog system building show that the proposed privacy-preserving mechanism achieves better utilities while remaining safe under various privacy attacks compared to the baselines. The data and code are released at https://github.com/wyshi/lm_privacy to facilitate future research.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “capabilities, most prominent solutions are based on differentially private (DP) learning (Kerrigan et al., 2020; Shi et al., 2021).”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “To address these privacy concerns, there is a growing body of literature that aims to create privacy-preserving language models [64, 2, 56, 98, 84, 40, 79].”
Submix: Practical Private Prediction for Large-Scale Language Models (2022, methodology): “More recently, Shi et al. (2021) explored an alternative approach called selective differential privacy , where the privacy guarantee only applies to blocks of text in the training corpus that are deemed sensitive, e.g. , addresses and phone numbers.”
Provably Confidential Language Modelling (2022, methodology): “In addition, the S-DP deﬁnition will not be useful for us we do not know how to deﬁne a conﬁdentiality parameter speciﬁc for each x or Bayesian conﬁdentiality parameter for each µ 2.”
Are Large Pre-Trained Language Models Leaking Your Personal Information? (2022, background): “To mitigate privacy leakage, there is a growing interest in making PLMs privacy-preserving (Anil et al., 2021; Li et al., 2022; Yu et al., 2021; Shi et al., 2021; Hoory et al., 2021).”