Submix: Practical Private Prediction for Large-Scale Language Models
By Antonio A. Ginart, L. V. D. van der Maaten, James Y. Zou, Chuan Guo. 2022
33 citations recorded by the source corpus.
Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim. This is a vulnerability that can compromise the privacy of the model's training data. In this work, we introduce SubMix: a practical protocol for private next-token prediction designed to prevent privacy violations by language models that were fine-tuned on a private corpus after pre-training on a public corpus. We show that SubMix limits the leakage of information that is unique to any individual user in the private corpus via a relaxation of group differentially private prediction. Importantly, SubMix admits a tight, data-dependent privacy accounting mechanism, which allows it to thwart existing data-extraction attacks while maintaining the utility of the language model. SubMix is the first protocol that maintains privacy even when publicly releasing tens of thousands of next-token predictions made by large transformer-based models such as GPT-2.
Subfields: Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Safety, Robustness & Interpretability
- Edward Raff
- Stella Biderman
- Gautam Kamath
- Sergey Yekhanin
- Richard S. Zemel
- Oskar van der Wal
- Arturs Backurs
- Rahul Gupta
Citations in the corpus
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, methodology): “While there are several reasonable definitions of memorization, we use the one from Carlini et al. (2021) as it has received considerable attention in the literature (Yoon & Lee, 2021; Huang et al., 2022; Ginart et al., 2022; Ippolito et al., 2022; Biderman et al., 2023).”
- Differentially Private Decoding in Large Language Models (2022, background): “To our best knowledge, prior to this work, only Ginart et al. (2022) study DP prediction for LLMs; the authors propose an ensemble-based method which significantly increases computational and storage costs.”
- Differentially Private Fine-tuning of Language Models (2021, background): “A simultaneous work of Ginart et al. (2022) investigates private prediction (rather than learning) for next-token prediction.”