Differentially Private Fine-tuning of Language Models

By Da Yu, Saurabh Naik, Arturs Backurs, S. Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz and 2 more. 2021

517 citations recorded by the source corpus.

We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning. Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the utility of private models approaches that of non-private models. For example, on the MNLI dataset we achieve an accuracy of $87.8\%$ using RoBERTa-Large and $83.5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6.7$. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of $90.2\%$. Our findings are similar for natural language generation tasks. Privately fine-tuning with DART, GPT-2-Small, GPT-2-Medium, GPT-2-Large, and GPT-2-XL achieve BLEU scores of 38.5, 42.0, 43.1, and 43.8 respectively (privacy budget of $\epsilon = 6.8,\delta=$ 1e-5) whereas the non-private baseline is $48.1$. All our experiments suggest that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Health Informatics, Large Language Models, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Health Informatics

In Large Language Models

In Privacy & Security

Citations in the corpus

Foundation Models and Fair Use (2023, background): “Recent works have developed substantial improvements to address this drawback through the use of better software primitives (Anil et al., 2021) and training techniques (Li et al., 2021; Yu et al., 2021; De et al., 2022; Sander et al., 2022).”
What Does it Mean for a Language Model to Preserve Privacy? (2022, background): “To address these privacy concerns, there is a growing body of literature that aims to create privacy-preserving language models [64, 2, 56, 98, 84, 40, 79].”
Are Large Pre-Trained Language Models Leaking Your Personal Information? (2022, background): “To mitigate privacy leakage, there is a growing interest in making PLMs privacy-preserving (Anil et al., 2021; Li et al., 2022; Yu et al., 2021; Shi et al., 2021; Hoory et al., 2021; Brown et al., 2022) by training PLMs with differential privacy guarantees (Dwork et al., 2006; Dwork, 2008) or…”
Unlocking High-Accuracy Differentially Private Image Classification through Scale (2022, methodology): “…has been recently used successfully when privately ﬁne-tuning language models (Li et al., 2021; Yu et al., 2021a), and it has also been shown to improve the performance of DP-SGD on both CIFAR-10 (Abadi et al., 2016; Tramèr and Boneh, 2021; Yu et al., 2021b) and ImageNet (Kurakin et al., 2022).”
TAN without a burn: Scaling Laws of DP-SGD (2022, methodology): “Yu et al. (2021b) propose to use low-rank reparametrization of the weight matrices to diminish the computational cost of accessing per-sample gradients.”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, methodology): “, has been used in many previous works (Zhou et al., 2020; Anil et al., 2021; Li et al., 2022b; Yu et al., 2021) and is probably most commonly used by practitioners, who “privatize” for DP training their best performing optimizer from a non-private setting.”
Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2022, background): “…data (data preprocessing) (Aura et al., 2006; Dernoncourt et al., 2017; Lison et al., 2021; Kandpal et al., 2022) or attempt to design algorithms that ensure differential privacy (DP) (Dwork, 2008; Dwork et al., 2006; Abadi et al., 2016; Anil et al., 2021; Li et al., 2022; Yu et al., 2022).”
Machine Learning for Synthetic Data Generation: a Review (2023, unknown): “While synthetic data promises a level of anonymity, there is an ongoing debate about the extent to which this data can be de-anonymized.”
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy (2022, background): “There is also a newly emerging line of works exploring differential-privacy (DP)-based definitions (Zhao et al., 2022; Stock et al., 2022), which relate to document-level DP guarantees in language modelling (Yu et al., 2021).”
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023, methodology): “…single training sample can have on the resulting model and has been shown to successfully protect models against membership inference attacks (Carlini et al., 2021a) and has recently also successfully been applied to training language models (Yu et al., 2022; Li et al., 2022; Mireshghallah et al.).”