Gender-preserving Debiasing for Pre-trained Word Embeddings

By Masahiro Kaneko, Danushka Bollegala. 2019

138 citations recorded by the source corpus.

Word embeddings learnt from massive text collections have demonstrated significant levels of discriminative biases such as gender, racial or ethnic biases, which in turn bias the down-stream NLP applications that use those word embeddings. Taking gender-bias as a working example, we propose a debiasing method that preserves non-discriminative gender-related information, while removing stereotypical discriminative gender biases from pre-trained word embeddings. Specifically, we consider four types of information: feminine, masculine, gender-neutral and stereotypical, which represent the relationship between gender vs. bias, and propose a debiasing method that (a) preserves the gender-related information in feminine and masculine words, (b) preserves the neutrality in gender-neutral words, and (c) removes the biases from stereotypical words. Experimental results on several previously proposed benchmark datasets show that our proposed method can debias pre-trained word embeddings better than existing SoTA methods proposed for debiasing word embeddings while preserving gender-related but non-discriminative information.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…Davidson et al. (2019); Dev et al. (2019); Garimella et al. (2019); Garg et al. (2019); Huang et al. (2019); James-Sorenson and AlvarezMelis (2019); Kaneko and Bollegala (2019); Liu et al. (2019); Karve et al. (2019); Nozza et al. (2019); Prabhakaran et al. (2019); Romanov et al. (2019); Sap et…”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “Particular attention has been paid to uncovering, analyzing, and removing gender biases in word embeddings (Basta et al., 2019; Kaneko and Bollegala, 2019; Zhao et al., 2019, 2018b; Bolukbasi et al., 2016).”
Multi-Dimensional Gender Bias Classification (2020, background): “For example, statistical gender biases are rampant in word embeddings (Jurgens et al., 2012; Bolukbasi et al., 2016; Caliskan et al., 2017; Garg et al., 2018; Zhao et al., 2018b; Basta et al., 2019; Chaloner and Maldonado, 2019; Du et al., 2019; Gonen and Goldberg, 2019; Kaneko and Bollegala, 2019; ”
Intersectional Bias in Causal Language Models (2021, background): “g.[28]), we also advocate for social approaches: study of the effects of bias on at-risk communities; developer and end-user risk education; and inclusion in language model design and evaluation of members of minority groups subject to bias.”
Detect All Abuse! Toward Universal Abusive Language Detection Models (2020, unknown): “The gender debiasing embedding (Kaneko and Bollegala, 2019) is applied.”
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research (2020, background): “Existing studies mostly focus on identifying gender-bias in context-independent word representations such as GloVe Bolukbasi et al. [11], Kaneko & Bollegala [89], Kumar et al.[107].”
Theories of “Gender” in NLP Bias Research (2022, unknown): “08975 [72] Masahiro Kaneko and Danushka Bollegala.”
A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations (2019, unknown): “…dump of English Wikipedia (represented by HSR-GloVe), and compare the performance of our al-gorithm with previous post-processing methods including the hard-debiasing method (Hard-GloVe) (Bolukbasi et al. 2016) and the gender-preserving debiasing method (GP-GloVe) (Kaneko and Bollegala 2019).”
deb2viz: Debiasing gender in word embedding data using subspace visualization (2020, background): “Thus, in contrast to our output, GP- [10] and GN- [11] makes inaccuracies with pre-trained word2vec model more than with GloVe [17].”
Debiasing Gender biased Hindi Words with Word-embedding (2019, background): “Recently, Kaneko and Bollegala [15] propose a novel approach and formulate the problem of debiasing by defining four mutually disjoint and exhaustive sets of words, words which are definitively female, definitively male, gender-neutral and stereotypical.”
Benchmarking Intersectional Biases in NLP (2022, background): “One body of work has focused on debiasing static word embeddings (Bolukbasi et al., 2016; Zhao et al., 2020, 2018; Kaneko and Bollegala, 2019; Ravfogel et al., 2020).”
Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation (2020, methodology): “Kaneko and Bollegala (2019) tackles this problem by adopting an encoder-decoder model to re-embed word embeddings.”