Privacy-preserving Neural Representations of Text

By Maximin Coavoux, Shay B. Cohen. 2018

129 citations recorded by the source corpus.

This article deals with adversarial attacks towards deep learning systems for Natural Language Processing (NLP), in the context of privacy protection. We study a specific type of attack: an attacker eavesdrops on the hidden representations of a neural text classifier and tries to recover information about the input text. Such scenario may arise in situations when the computation of a neural network is shared across multiple devices, e.g. some hidden representation is computed by a user’s device and sent to a cloud-based model. We measure the privacy of a hidden representation by the ability of an attacker to predict accurately specific private information from it and characterize the tradeoff between the privacy and the utility of neural representations. Finally, we propose several defense methods based on modified training objectives and show that they improve the privacy of neural representations.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020, background): “While adverserial methods showed impressive performance in various machine learning tasks, and were applied for the goal of removal of sensitive information (Elazar and Goldberg, 2018; Coavoux et al., 2018; Resheff et al., 2019; Barrett et al., 2019), they are notoriously hard to train.”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “A variety of papers have begun to address countermeasures for predictive biases (e.g. Li et al. (2018); Elazar and Goldberg (2018); Coavoux et al. (2018)).”
Information Leakage in Embedding Models (2020, methodology): “ through adversarial training [17, 74] as detailed in Section 7. The same approach as been applied in NLP models to remove sensitive attributes from the representation for privacy or fairness concern [9, 18, 38]. Another approach for removing sensitive attributes is through directly minimizing the m”
Overlearning Reveals Sensitive Attributes (2019, methodology): “Techniques include adversarial training (Edwards & Storkey, 2016), which has been applied to census and health records (Xie et al., 2017), text (Li et al., 2018; Coavoux et al., 2018; Elazar & Goldberg, 2018), images (Hamm, 2017) and sensor data of wearables (Iwasawa et al., 2016).”
Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels (2021, unknown): “Our work is closely related to (Coavoux et al., 2018) and (Li et al., 2018).”
The Hitchhiker’s Guide to Computational Linguistics in Suicide Prevention (2021, background): “…how this data-extraction process may reveal private information about the authors of training texts or individuals for which the model makes predictions, including their names, contact information, and sociodemographic background (Carlini et al., 2020; Coavoux et al., 2018; Li et al., 2018).”
Large Language Models Can Be Strong Differentially Private Learners (2021, background): “Orthogonally, many works considered training language models that satisfy empirical notions of privacy (Xu et al., 2021; Coavoux et al., 2018; Mireshghallah et al., 2021; Melamud & Shivade, 2019).”
Five sources of bias in natural language processing (2021, background): “Some researchers have pointed out how NLP technologies can be used for harmful purposes, such as suppressing dissenters (Bamman et al., 2012; Zhang et al., 2014), compromising privacy/anonymity (Coavoux et al., 2018; Grouin et al., 2015), or profiling (Jurgens et al., 2017; Wang et al., 2018).”
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset (2022, background): “But a number of recent papers have demonstrated that large generative models output memorized content [26, 111, 32, 25, 69] even with deduplication [27].”
You Don’t Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers’ Private Personas (2022, background): “Adversarial training is widely used to protect sensitive features in natural language processing (Elazar and Goldberg, 2018; Coavoux et al., 2018; Li et al., 2018).”