Learning Differentially Private Language Models Without Losing Accuracy

By H. Brendan McMahan, Daniel Ramage, Kunal Talwar, Li Zhang. 2017

129 citations recorded by the source corpus.

We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees without sacrificing predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes"large step"updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.

Subfields: Artificial Intelligence, Large Language Models, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Privacy & Security

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, methodology): “DPSGD [22] # # MNIST, CIFAR-10 Image Classification w/ DNN DP LSTM [77] # # Reddit Posts Language Model w/ LSTMs DP LOR [78] # # Artificial Data Logistic Regression DP ERM [79] # # Adult, KDD-99 Classification w/ ERM DP GAN [80] # # MNIST, MIMIC-III Data Generation w/ GAN DP GM [81] # # MNIST, CDR, ”
AdaCliP: Adaptive Clipping for Private SGD (2019, background): “Differential privacy for machine learning has been studied in various models including models with convex objectives [10, 11, 12] and more recently deep learning methods [13, 14, 15].”
Federated Learning Of Out-Of-Vocabulary Words (2019, background): “Such risk can be mitigated, usually with some accuracy cost, using techniques including differential privacy (McMahan et al., 2018).”
Federated Machine Learning (2019, neutral): “s require participants’ data to be secretly-shared among non-colluding servers. DifferentialPrivacy.AnotherlineofworkusetechniquesDifferentialPrivacy[18]ork-Anonymity [63] for data privacy protection [1, 12, 42, 61]. The methods of differential privacy, k-anonymity, and diversification [3] involve i”
Protection Against Reconstruction and Its Applications in Private Federated Learning (2018, methodology): “[55] note, increasing this batch size has negligible effect on the accuracy of the centralized model, so that we report results (following [55]) that use this inflated batch size estimate from a population of size N = 10,000,000.”
Federated Learning for Mobile Keyboard Prediction (2018, methodology): “Language models have been trained using the federated algorithm combined with differential privacy [17, 18].”
Differentially Private Federated Learning: A Client Level Perspective (2017, background): “Experimental setups however differ and [6] also includes element-level privacy measures.”
Scalable Private Learning with PATE (2018, background): “For example, models trained on people’s personal messages or detailed medical information can offer invaluable insights into real-world language usage or the diagnoses and treatment of human diseases (McMahan et al., 2017; Liu et al., 2017).”
Auditing Data Provenance in Text-Generation Models (2018, methodology): “n specific hidden units in the model and thus involves more memorization. 6 LIMITATIONS OF AUDITING Modelstrainedonaverylargenumberofusers. In some industrial implementation of text-generation models [22, 23], the number of users is on the scale of millions. Performance of our auditor starts to drop”
Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation (2020, background): “Or, to help build better predictive keyboards, respondents may send LDP reports about the words they type annotated by their software’s preferred-language settings (e.g., EN-US, EN-IN, CHS, or CHT); such partitioned LDP reporting is realistic and has been deployed in practice [7], [36].”
Differentially Private Model Publishing for Deep Learning (2019, background): “The characteristics of input data, for example, dependency among training instances or dependency in the presence of training instances can render a differentially private mechanism ineffective for protecting the privacy of individuals [24], [25], [28].”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “To address these privacy concerns, there is a growing body of literature that aims to create privacy-preserving language models [64, 2, 56, 98, 84, 40, 79].”
Distributed Learning without Distress: Privacy-Preserving Empirical Risk Minimization (2018, background): “The goal of distributed machine learning (also referred to as federated learning [36]) is to enable a group of independent data owners to develop a model from their combined data without exposing that data to others.”
FLAME: Taming Backdoors in Federated Learning (2022, methodology): “We compare FLAME to existing defenses: Krum [9], FoolsGold [22], Auror [51], Adaptive Federated Averaging (AFA) [42], Median [60] and a generalized differential privacy (DP) approach [7, 40].”