Adversarial Removal of Demographic Attributes from Text Data

By Yanai Elazar, Yoav Goldberg. 2018

352 citations recorded by the source corpus.

Recent advances in Representation Learning and Adversarial Training seem to succeed in removing unwanted features from the learned representation. We show that demographic information of authors is encoded in—and can be recovered from—the intermediate representations learned by text-based neural classifiers. The implication is that decisions of classifiers trained on textual data are not agnostic to—and likely condition on—demographic attributes. When attempting to remove such demographic information using adversarial training, we find that while the adversarial component achieves chance-level development-set accuracy during training, a post-hoc classifier, trained on the encoded sentences from the first part, still manages to reach substantially higher classification accuracies on the same data. This behavior is consistent across several tasks, demographic properties and datasets. We explore several techniques to improve the effectiveness of the adversarial component. Our main conclusion is a cautionary one: do not rely on the adversarial training to achieve invariant representation to sensitive features.

Subfields: Artificial Intelligence, General Social Sciences, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

WinoGrande (2019, methodology): “This approach is also reminiscent of recent work in NLP on adversarial learning (Chen and Cardie 2018; Belinkov et al. 2019; Elazar and Goldberg 2018).”
Evaluating Gender Bias in Machine Translation (2019, background): “While recent research has shown that getting rid of such biases may prove to be very challenging (Elazar and Goldberg, 2018; Gonen and Goldberg, 2019), we hope that this work will serve as a ﬁrst step for developing more gender-balanced MT models.”
Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models (2019, background): “[Zhang et al., 2018] introduce an adversarial setup where adversary tries to identify a protected variable based on a predicted variable, while [Elazar and Goldberg, 2018] show that adversarial removal leads to leakage of bias.”
Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, methodology): “A key difference between the above and our approach is that we use an ofﬂine discriminator and perform optimization based on this discriminator, which as suggested by Elazar & Goldberg (2018) may outperform adversarial training approaches.”
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2020, background): “Additionally, Elazar and Goldberg (2018) found that this technique does not completely remove all the information from the learned representation.”
CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, background): “bility, debiasing is a rising sub-ﬁeld that deals with creating models and language representations that are unaffected by unwanted biases that might exist in the data (Kiritchenko and Mohammad 2018; Elazar and Goldberg 2018; Gonen and Goldberg 2019; Ravfogel et al. 2020). DNNs are as good as the tr”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “…embed-dings and contextual word representations (Boluk-basi et al., 2016; Zhao et al., 2018b; Yang and Feng, 2020), though recent work has questioned the ef-ﬁcacy of these debiasing techniques in removing both grammatical and societal biases (Elazar and Goldberg, 2018; Gonen and Goldberg, 2019).”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, background): “Other important aspects such as model/data trans-parency (Mitchell et al., 2019; Bender and Fried-man, 2018) and privacy preservation (Reddy and Knight, 2016; Elazar and Goldberg, 2018; Li et al., 2018) are also not covered in this literature survey.”
DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks (2020, background): “While not motivating private collaborative inference, a few recent works [23, 24] have attempted the related problem of attribute leakage [24, 25, 26, 27, 28] by focusing on adversarial representation learning (ARL).”
Differential Privacy Has Disparate Impact on Model Accuracy (2019, methodology): “To simplify the task and speed up convergence, we used a technique inspired by [15] and with probability 90% appended to each tweet a special emoji associated with the tweet’s class and subgroup.”
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification (2019, background): “In addition, several works, such as [10], [1], [6] and [5] provide techniques intending to mitigate bias in machine learning models.”
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020, methodology): “We follow the setup of Elazar and Goldberg (2018) which used a twitter dataset, collected by Blodgett et al. (2016), where each tweet is associated with “race” information and a sentiment which was determined by their belonging to some emoji group.”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “The findings from Elazar and Goldberg (2018) suggest that even with adversarial training, internal representations still retain traces of demographic information.”
Measuring and Reducing Gendered Correlations in Pre-trained Models (2020, positive): “Another popular technique uses adversarial losses in order to remove demographic information from learned representations (Louizos et al., 2015; Edwards and Storkey, 2015; Beutel et al., 2017; Zhang et al., 2018; Elazar and Goldberg, 2018; Madras et al., 2018).”
Demoting Racial Bias in Hate Speech Detection (2020, methodology): “Adversarial training has been used to remove protected features from word embeddings (Xie et al., 2017; Zhang et al., 2018) and intermediate representations for both texts (Elazar and Goldberg, 2018; Zhang et al., 2018) and images (Edwards and Storkey, 2015; Wang et al., 2018).”
Information Leakage in Embedding Models (2020, methodology): “ through adversarial training [17, 74] as detailed in Section 7. The same approach as been applied in NLP models to remove sensitive attributes from the representation for privacy or fairness concern [9, 18, 38]. Another approach for removing sensitive attributes is through directly minimizing the m”
Adversarial Removal of Gender from Deep Image Representations (2018, neutral): “In contrast to these works, we try to address removal of bias in the feature space, therefore we adopt and further develop the idea of leakage as an evaluation criteria, as proposed for the debiasing of text representations used by Elazar and Goldberg [8].”
There is Strength in Numbers: Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training (2020, background): “Similarly, Elazar and Goldberg (2018) found that adversarial training and gradient reversal does not remove demographic information such as age or gender, with this information still present in the de-biased sentence representations.”