Are Large Pre-Trained Language Models Leaking Your Personal Information?

By Jie Huang, Hanyin Shao, Kevin Chen-Chuan Chang. 2022

297 citations recorded by the source corpus.

Are Large Pre-Trained Language Models Leaking Your Personal Information? In this paper, we analyze whether Pre-Trained Language Models (PLMs) are prone to leaking personal information. Specifically, we query PLMs for email addresses with contexts of the email address or prompts containing the owner's name. We find that PLMs do leak personal information due to memorization. However, since the models are weak at association, the risk of specific personal information being extracted by attackers is low. We hope this work could help the community to better understand the privacy risk of PLMs and bring new insights to make PLMs safe.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Safety, Robustness & Interpretability

Citations in the corpus

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, methodology): “While there are several reasonable definitions of memorization, we use the one from Carlini et al. (2021) as it has received considerable attention in the literature (Yoon & Lee, 2021; Huang et al., 2022; Ginart et al., 2022; Ippolito et al., 2022; Biderman et al., 2023).”
The ROOTS Search Tool: Data Transparency for LLMs (2023, background): “…we now know that the data used for training LLMs may contain synthetic data (Dodge et al., 2021), privacy-infringing data (Carlini et al., 2020; Huang et al., 2022), incorrect language codes or and translations (Kreutzer et al., 2022), not to mention the ubiquitous issues with social biases…”
Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2022, background): “…Language Models (LMs) including Personally Identifiable Information (PII) such as names, phone numbers, and email addresses, and other information such as licensed code, private clinical notes, and 128-bit UUIDs (Carlini et al., 2021; Lee et al., 2022; Huang et al., 2022; Lehman et al., 2021).”
Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning (2024, background): “For instance, language models could breach individual privacy by outputting personal information such as social security numbers from the memorized data (Carlini et al., 2021; Huang et al., 2022).”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “In this section, we examine the range of existing defense methods against various attacks and vulnerabilities associated with LLMs 1 .”
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories (2022, background): “Recent work (Huang et al., 2022) shows that LMs memorize personal information available on the web, which has significant security issues.”
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, unknown): “They provide evidence that NLI test samples deviating from these patterns result in significantly lower scores compared to those adhering to them.”
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, background): “Huang et al. [118] found that LMs can leak personal information.”
Multi-step Jailbreaking Privacy Attacks on ChatGPT (2023, methodology): “Several studies (Lukas et al., 2023; Huang et al., 2022; Carlini et al., 2021) suggested that LMs tend to memorize their training data and partial private information might be recovered given specific prompts.”