Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them

By Hila Gonen, Yoav Goldberg. 2019

632 citations recorded by the source corpus.

Word embeddings are widely used in NLP for a vast range of tasks. It was shown that word embeddings derived from text corpora reflect gender biases in society. This phenomenon is pervasive and consistent across different word embedding models, causing serious concern. Several recent works tackle this problem, and propose methods for significantly reducing this gender bias in word embeddings, demonstrating convincing results. However, we argue that this removal is superficial. While the bias is indeed substantially reduced according to the provided bias definition, the actual effect is mostly hiding the bias, not removing it. The gender bias information is still reflected in the distances between “gender-neutralized” words in the debiased embeddings, and can be recovered from them. We present a series of experiments to support this claim, for two debiasing methods. We conclude that existing bias removal techniques are insufficient, and should not be trusted for providing gender-neutral modeling.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

In Human-Computer Interaction

Citations in the corpus

Language Models are Few-Shot Learners (2020, background): “Thus, mitigation work should not be approached purely with a metric driven objective to ‘remove’ bias as this has been shown to have blind spots [GG19, NvNvdG19] but in a holistic manner.”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…and Bhallamudi (2019); Brunet et al. (2019); Cao and Daumé (2019); Chaloner and Maldonado (2019); Dev and Phillips (2019); Ethayarajh et al. (2019); Gonen and Goldberg (2019); James-Sorenson and Alvarez-Melis (2019); Jumelet et al. (2019); Kaneko and Bollegala (2019); Karve et al. (2019); Kurita…”
Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function (2019, background): “But, this bias deﬁnition is shown to be incomplete by Gonen and Goldberg (2019).”
Evaluating Gender Bias in Machine Translation (2019, background): “This may be explained by German’s similarity to the English source language (Hawkins, 2015).”
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020, background): “However, follow-up work by Gonen and Goldberg (2019) shows that this method only hides the bias and does not remove it.”
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models (2021, background): “However, these offer superficial reductions in gender bias, and systematic prejudice was found to persist (Gonen and Goldberg, 2019).”
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, neutral): “First, as discussed in the literature (e.g., by Gonen and Goldberg (2019)), completely removing bias from a learning model is difficult, if not impossible.”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “…embed-dings and contextual word representations (Boluk-basi et al., 2016; Zhao et al., 2018b; Yang and Feng, 2020), though recent work has questioned the ef-ﬁcacy of these debiasing techniques in removing both grammatical and societal biases (Elazar and Goldberg, 2018; Gonen and Goldberg, 2019).”
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP (2021, background): “…exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhao et al., 2017; Rudinger et al., 2018; Gonen and Goldberg, 2019; Bordia and Bowman, 2019; Sheng et al., 2019; Basta et al., 2019; Nangia et al., 2020, i.a. ) and are prone to generating…”
Identifying and Reducing Gender Bias in Word-Level Language Models (2019, methodology): “It can be concluded different bias metrics show different kinds of bias (Gonen and Goldberg, 2019).”
Gender Bias in Contextualized Word Embeddings (2019, background): “However, Gonen and Goldberg (2019) argue that entirely removing bias is difﬁcult, if not impossible, and the gender bias information can be often recovered.”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, methodology): “While existing methods successfully mitigate bias with respect to projection onto the gender subspace in some degrees, Gonen and Goldberg (2019) show that gender bias based on more subtle metrics such as cluster bias still exist.”
Evaluating the Underlying Gender Bias in Contextualized Word Embeddings (2019, methodology): “A note to be considered, is that the lists we used in our experiments (and obtained from Bolukbasi et al. (2016) and Gonen and Goldberg (2019)) may contain words that are missing in our corpus and so we can not obtain contextualized embeddings for them.”
Assessing Social and Intersectional Biases in Contextualized Word Representations (2019, methodology): “ure on methods to de-bias word embeddings, including post-processing methods [2] and techniques that use constraints during training [34, 32]. Although these methods are promising, Gonen and Goldberg [13] show that the gender bias encoded in word embeddings is mostly hidden from the deﬁned metric of”