Mitigating Gender Bias in Natural Language Processing: Literature Review

By Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai Elsherief, Jieyu Zhao, Diba Mirza, Elizabeth M. Belding-Royer, Kai-Wei Chang, William Yang Wang. 2019

648 citations recorded by the source corpus.

As Natural Language Processing (NLP) and Machine Learning (ML) tools rise in popularity, it becomes increasingly vital to recognize the role they play in shaping societal biases and stereotypes. Although NLP models have shown success in modeling various applications, they propagate and may even amplify gender bias found in text corpora. While the study of bias in artificial intelligence is not new, methods to mitigate gender bias in NLP are relatively nascent. In this paper, we review contemporary studies on recognizing and mitigating gender bias in NLP. We discuss gender bias based on four forms of representation bias and analyze methods recognizing gender bias. Furthermore, we discuss the advantages and drawbacks of existing gender debiasing methods. Finally, we discuss future studies for recognizing and mitigating gender bias in NLP.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…(2019); Kurita et al. (2019); Mayfield et al. (2019); Pujari et al. (2019); Romanov et al. (2019); Ruane et al. (2019); Sedoc and Ungar (2019); Sun et al. (2019); Zmigrod et al. (2019); Hutchinson et al. (2020); Papakyriakopoulos et al. (2020); Ravfogel et al. (2020); Strengers et al. (2020);…”
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021, neutral): “There have been multiple attempts to understand social biases in language processing (Sheng et al., 2021; Caliskan et al., 2017), quantify them (Rudinger et al., 2018; Webster et al., 2018; De-Arteaga et al., 2019), and mitigate them (Zhao et al., 2019; Ravfo-gel et al., 2020; Sun et al., 2019).”
Societal Biases in Language Generation: Progress and Challenges (2021, background): “Sun et al. (2019) present a focused survey Table 1: Existing bias studies on different demographic dimensions in various NLG tasks: autocomplete generation, dialogue generation, machine translation (MT), and text re-writing. on mitigating gender biases and Shah et al. (2020) categorize sources of…”
Re-contextualizing Fairness in NLP: The Case of India (2022, background): “NLP fairness research focuses on gender (Bolukbasi et al., 2016; Sun et al., 2019; Zhao et al., 2017) & racial biases (Sap et al.”
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation (2021, methodology): “Following current literature that studies gender bias in models [2, 35], we obtain a list for male and female identifying tokens as: male tokens he, him, his, himself, man, men, he’s, boy, boys and female tokens she, her, hers, herself, woman, women, she’s, girl and girls.”
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models (2021, methodology): “free from unwanted bias and whilst it is true that they exhibit less gender bias than static word embeddings (Sun et al., 2019), they are by no means immune to this problem (Lu et al.”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “Recently, the NLP community has focused on exploring gender bias in NLP systems (Sun et al., 2019), uncovering many gender disparities and harmful biases in algorithms and text (Cao and Daum´e III 2020; Chang et al. 2019; Chang and McKeown 2019; Costa-juss ` a 2019; Du et al. 2019; Emami et al.…”
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, neutral): “Our work also touches upon the fairness literature (e.g., Bolukbasi et al., 2016; Dev et al., 2020; Chang et al., 2019; Blodgett et al., 2020; Sun et al., 2019).”
Multi-Dimensional Gender Bias Classification (2020, background): “Gender affects myriad aspects of NLP, including corpora, tasks, algorithms, and systems (Chang et al., 2019; Costa-jussà, 2019; Sun et al., 2019).”
A Survey on Bias and Fairness in Natural Language Processing (2022, neutral): “This is a very narrow view, and Gender Bias Evaluation Test-Sets (GBETs) introduced by Sun et al. (2019) provides a solution to this problem.”
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, background): “…resolution (Zhao et al., 2018a), named entity recognition (Mehrabi et al., 2020), machine translation (Font and Costa-jussà, 2019), word embedding (Bolukbasi et al., 2016; Zhao et al., 2018b; Zhou et al., 2019), as well as surveys (Sun et al., 2019; Blodgett et al., 2020; Mehrabi et al., 2021).”
Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition (2019, background): “Some progress has been made on the analysis of gender stereotyping in different natural language processing (NLP) components, such as word embedding, co-reference resolution, machine translation and sentence encoders [2].”
Towards Debiasing Sentence Representations (2020, background): “Systems that incorporate these embeddings can further amplify biases [24, 34, 3] and unfairly discriminate against users, particularly those from disadvantaged social groups.”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “The “bias” discussed in one paper may therefore be quite different than that in another.2 A shared definition and framework of predictive bias would unify these efforts, provide a common terminology, help to identify underlying causes, and allow coordination of countermeasures (Sun et al., 2019).”
UNQOVERing Stereotypical Biases via Underspecified Questions (2020, background): “Unfortunately, these representations learn stereotypes often enmeshed in the massive body of text used to train them (Sun et al., 2019).”
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge (2020, background): “…has also been extended with works looking at ways of incorporating raw text and symbolic KGs into a shared embedding space (Riedel et al., 2013; Verga et al., 2016), to be jointly reasoned over (Sun et al., 2018, 2019a), or to treat text as a replacement for a knowledge base (Dhingra et al., 2019).”