Large-Scale Differentially Private BERT

By Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi. 2021

162 citations recorded by the source corpus.

In this work, we study the large-scale pretraining of BERT-Large with differentially private SGD (DP-SGD). We show that combined with a careful implementation, scaling up the batch size to millions (i.e., mega-batches) improves the utility of the DP-SGD step for BERT; we also enhance its efficiency by using an increasing batch size schedule. Our implementation builds on the recent work of [SVK20], who demonstrated that the overhead of a DP-SGD step is minimized with effective use of JAX [BFH+18, FJL18] primitives in conjunction with the XLA compiler [XLA17]. Our implementation achieves a masked language model accuracy of 60.5% at a batch size of 2M, for $\epsilon = 5.36$. To put this number in perspective, non-private BERT models achieve an accuracy of $\sim$70%.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Safety, Robustness & Interpretability

Citations in the corpus

Quantifying Memorization Across Neural Language Models (2022, background): “, 2016) generally suffer from expensive computation, slow convergence, and poor model utility, despite recent advances (Anil et al., 2021).”
Foundation Models and Fair Use (2023, background): “Recent works have developed substantial improvements to address this drawback through the use of better software primitives (Anil et al., 2021) and training techniques (Li et al., 2021; Yu et al., 2021; De et al., 2022; Sander et al., 2022).”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “To address these privacy concerns, there is a growing body of literature that aims to create privacypreserving language models [64, 2, 56, 98, 84, 40, 79].”
Unlocking High-Accuracy Differentially Private Image Classification through Scale (2022, methodology): “As observed in several recent papers (Anil et al., 2021; Dörmann et al., 2021; Hoory et al., 2021; Li et al., 2021; Luo et al., 2021), increasing the batch size can signiﬁcantly improve the privacy-utility trade-oﬀ of DP-SGD.”
Emergent and Predictable Memorization in Large Language Models (2023, methodology): “However, such methods have been observed to hurt performance during pretraining [Anil et al., 2021], and are therefore not popular among people who train large language models.”
Large Language Models Can Be Strong Differentially Private Learners (2021, background): “Anil et al. (2021) pretrained BERT under global DP on datasets with hundreds of millions of examples.”
The MiniPile Challenge for Data-Efficient Language Models (2023, background): “structurally different architectures [16, 63, 32], pre-training schemes [50, 41], optimizers schemes [3, 26, 24], differential privacy [2, 5], mechanistic interpretability [15, 46, 6], etc.”
Differentially Private Fine-tuning of Language Models (2021, background): “To the best of our knowledge, we are the ﬁrst to ﬁne-tune GPT-2-Large using differential privacy, the largest model trained thus far using DP.”
Are Large Pre-Trained Language Models Leaking Your Personal Information? (2022, methodology): “, 2021) and implemented in (Anil et al., 2021), we can train the model with differentially private stochastic gradient descent (DP-SGD) algorithm (Abadi et al.”
Security and Privacy Challenges of Large Language Models: A Survey (2024, unknown): “In training, following the process of [11] and the implementation of [186], the differentially private stochastic gradient descent (DP-SGD) algorithm [173] can be employed to ensure privacy of training data during the training process [11], [187].”
TAN without a burn: Scaling Laws of DP-SGD (2022, methodology): “We follow the standard practice of not counting hyper-parameter search towards the privacy budget (Li et al., 2021; Anil et al., 2021).”
Provably Confidential Language Modelling (2022, methodology): “Anil et al. (2021) pre-trained BERT under DP on datasets with hundreds of millions of examples.”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, methodology): “Anil et al. (2021) report an opposite effect that scale invariant layers like BatchNorm have on the utility: the Gaussian noise injected into the gradients increases the Frobenious norm of the weights during training, which in turn reduces the magnitude of the gradients and slows down the training…”
Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2022, background): “, 2022), pretraining LMs with DP still suffers from substantial performance gap, expensive computation, and slow convergence (Anil et al., 2021).”
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy (2022, background): “However, in practice, these techniques result in worse generative models (Anil et al., 2021)—thus, no state-of-the-art, large, language models are trained with DP.”
Individual Privacy Accounting via a Renyi Filter (2020, background): “Recent large-scale experiments on differentially private training of language models [2] similarly point to larger batches leading to higher utility, and we believe this phenomenon likely holds in many other settings and is worth further exploration.”