Differentially Private Decoding in Large Language Models
By Jimit Majmudar, Christophe Dupuy, Charith Peris, S. Smaili, Rahul Gupta, Richard S. Zemel. 2022
47 citations recorded by the source corpus.
Recent large-scale natural language processing (NLP) systems use a pre-trained Large Language Model (LLM) on massive and diverse corpora as a headstart. In practice, the pre-trained model is adapted to a wide array of tasks via fine-tuning on task-specific datasets. LLMs, while effective, have been shown to memorize instances of training data thereby potentially revealing private information processed during pre-training. The potential leakage might further propagate to the downstream tasks for which LLMs are fine-tuned. On the other hand, privacy-preserving algorithms usually involve retraining from scratch, which is prohibitively expensive for LLMs. In this work, we propose a simple, easy to interpret, and computationally lightweight perturbation mechanism to be applied to an already trained model at the decoding stage. Our perturbation mechanism is model-agnostic and can be used in conjunction with any LLM. We provide theoretical analysis showing that the proposed mechanism is differentially private, and experimental results showing a privacy-utility trade-off.
Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yulia Tsvetkov
- Vidhisha Balachandran
- Antonios Anastasopoulos
- Sachin Kumar
- Lucille Njoo
- Wael Hamza
- Mohit Bansal
- Sohee Yang
In Large Language Models
- Vidhisha Balachandran
- Yulia Tsvetkov
- Premkumar Natarajan
- Sohee Yang
- Anna Rumshisky
- Gökhan Tür
- Wael Hamza
- Rahul Gupta
In Safety, Robustness & Interpretability
- Sachin Kumar
- Vidhisha Balachandran
- Lucille Njoo
- Antonios Anastasopoulos
- Yulia Tsvetkov
- Premkumar Natarajan
- Anna Rumshisky
- Rahul Gupta
Citations in the corpus
- Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “, 2022) Privacy Differentially private decoding (Majmudar et al., 2022) Misinformation / Factuality Autoregressive factual error control(King et al.”
- A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “LLMs have gained traction in forensic investigations, offering novel approaches for analyzing digital evidence.”
- AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model (2022, background): “While posthoc privacy auditing (Jagielski et al., 2020; Jayaraman & Evans, 2019; Nasr et al., 2021) and leakage mitigation processes (Majmudar et al., 2022) can be used to limit the risk, users should in general be mindful when making decisions on how and when large language models such as AlexaTM…”
- Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2022, methodology): “We evaluate DP Decoding only on the 4 dialogue tasks because the decoding strategy cannot be applied for performing the classification tasks which is evaluated by utilizing a verbalizer.”