A Survey on Gender Bias in Natural Language Processing

By Karolina Stańczak, Isabelle Augenstein. 2021

168 citations recorded by the source corpus.

Language can be used as a means of reproducing and enforcing harmful stereotypes and biases and has been analysed as such in numerous research. In this paper, we present a survey of 304 papers on gender bias in natural language processing. We analyse definitions of gender and its categories within social sciences and connect them to formal definitions of gender bias in NLP research. We survey lexica and datasets applied in research on gender bias and then compare and contrast approaches to detecting and mitigating gender bias. We find that research on gender bias suffers from four core limitations. 1) Most research treats gender as a binary variable neglecting its fluidity and continuity. 2) Most of the work has been conducted in monolingual setups for English or other high-resource languages. 3) Despite a myriad of papers on gender bias in NLP methods, we find that most of the newly developed algorithms do not test their models for bias and disregard possible ethical considerations of their work. 4) Finally, methodologies developed in this line of research are fundamentally flawed covering very limited definitions of gender bias and lacking evaluation baselines and pipelines. We suggest recommendations towards overcoming these limitations as a guide for future research.

Subfields: Artificial Intelligence, Gender Studies, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dirk Hovy
2. Federico Bianchi
3. Debora Nozza
4. Yonatan Belinkov
5. Margaret Mitchell
6. Yoav Goldberg
7. Anna Rogers
8. Dragomir R. Radev

In Gender Studies

1. Alexandra Sasha Luccioni

In Natural Language Processing

1. Dirk Hovy
2. Federico Bianchi
3. Debora Nozza
4. Alexandra Sasha Luccioni
5. Anna Rogers
6. Stella Biderman
7. Aleksandra Piktus
8. Shanya Sharma Sharma

In AI Ethics, Fairness & Society

1. Margaret Mitchell
2. Alexandra Sasha Luccioni
3. Debora Nozza
4. Yacine Jernite
5. Zeerak Talat
6. Svetlana Kiritchenko
7. Michael Wiegand
8. Mladen Karan

Citations in the corpus

• The ROOTS Search Tool: Data Transparency for LLMs (2023, background): “…et al., 2021), privacy-infringing data (Carlini et al., 2020; Huang et al., 2022), incorrect language codes or and translations (Kreutzer et al., 2022), not to mention the ubiquitous issues with social biases (Blodgett et al., 2020; Field et al., 2021; Stanczak and Augenstein, 2021, among others).”
• You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, methodology): “Considering gender bias, Stanczak and Augenstein (2021) show that existing methods (1) largely avoid eth-182 ical considerations or evaluations of gender bias, (2) focus primarily on binary gender treatment, in mostly Anglo-centric settings, and (3) employ limited or flawed evaluation methodologies.”
• Undesirable biases in NLP: Averting a crisis of measurement (2022, background): “Despite all this work, conceptually, bias remains poorly understood (Blodgett et al., 2020; Dev et al., 2022; Stanczak & Augenstein, 2021; Talat et al., 2022, i.a.).”
• Pipelines for Social Bias Testing of Large Language Models (2022, background): “An overview of bias in NLP has been presented in several work (Blodgett et al., 2020; Shah et al., 2020; Hovy and Prabhumoye, 2021; Sheng et al., 2021; Stanczak and Augenstein, 2021).”