Theories of “Gender” in NLP Bias Research

By Hannah Devinney, Jenny Björklund, Henrik Björklund. 2022

97 citations recorded by the source corpus.

The rise of concern around Natural Language Processing (NLP) technologies containing and perpetuating social biases has led to a rich and rapidly growing area of research. Gender bias is one of the central biases being analyzed, but to date there is no comprehensive analysis of how “gender” is theorized in the field. We survey nearly 200 articles concerning gender bias in NLP to discover how the field conceptualizes gender both explicitly (e.g. through definitions of terms) and implicitly (e.g. through how gender is operationalized in practice). In order to get a better idea of emerging trajectories of thought, we split these articles into two sections by time. We find that the majority of the articles do not make their theorization of gender explicit, even if they clearly define “bias.” Almost none use a model of gender that is intersectional or inclusive of nonbinary genders; and many conflate sex characteristics, social gender, and linguistic gender in ways that disregard the existence and experience of trans, nonbinary, and intersex people. There is an increase between the two time-sections in statements acknowledging that gender is a complicated reality, however, very few articles manage to put this acknowledgment into practice. In addition to analyzing these findings, we provide specific recommendations to facilitate interdisciplinary work, and to incorporate theory and methodology from Gender Studies. Our hope is that this will produce more inclusive gender bias research in NLP.

Subfields: Artificial Intelligence, Communication, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Natural Language Processing

Citations in the corpus

“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, background): “…that works that attempt to measure bias often run into inadequate or incomplete definitions of bias (Blodgett et al., 2020): for instance, Devinney et al. (2022) surveys nearly 200 articles regarding gender bias in NLP and finds that almost all of them do not clearly specify how…”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “Reify-ing essentialist categories can contribute to “existential harm” in which people are “portrayed in overly reductive terms” [186, p. 162], often from a Western or Eurocentric perspective [69].”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “162), often from a Western or eurocentric perspective [59].”
Bias and Fairness in Large Language Models: A Survey (2023, background): “For example, Devinney et al. (2022) argue that modeling gender as binary and immutable erases the identities of trans, nonbinary, and intersex people.”