Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies

By Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff M. Phillips, Kai-Wei Chang. 2021

214 citations recorded by the source corpus.

Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “…al., 2021b), there is a lack of standardization of fairness benchmarks, an understanding of what harms different bias measures in NLP relate to (Blodgett et al., 2020, 2021; Jacobs & Wallach, 2021), and coverage of identities in fluid, comprehensive ways (Cao & Daumé III, 2020; Dev et al., 2021a).”
GLM-130B: An Open Bilingual Pre-trained Model (2022, methodology): “, 2022)), could be used to generate synthetic text for harmful applications, such as telemarketing fraud, political propaganda, and personal harassment as is discussed in (Weidinger et al., 2021; Sheng et al., 2021; Dev et al., 2021).”
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, background): “5, 6 [28] Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff M Phillips, and Kai-Wei Chang.”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “In efforts to address the expression of such social biases, a number of bias evaluation benchmarks have been proposed (Dev et al., 2021b; Zhao et al., 2018; Cao and Daumé III, 2020).”
Socially Aware Bias Measurements for Hindi Language Representations (2021, background): “Furthermore, we acknowledge that our analysis of gender associated biases is limited to binary gender and our intrinsic evaluations require discrete categorizations (Dev et al., 2021b; Antoniak and Mimno, 2021).”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “We expect and hope it will evolve as research and engagement progress, particularly in terms of participatory and community-driven research methodologies (e.g., [68, 71, 89]).”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “We expect and hope it will evolve as research progresses, particularly in terms of user and community-driven research methodologies (e.g., [58, 61, 79]).”
Handling Bias in Toxic Speech Detection: A Survey (2022, background): “[23] provided a general overview of how non-binary individuals are at risk of erasure and misgendering at the hands of existing language models.”
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2020, background): “The concept of gender is a complex one (cf. Larson, 2017; Dev et al., 2021).”
This Prompt is Measuring : Evaluating Bias Evaluation in Language Models (2023, background): “…for example, he and she are almost always described as male and female pronouns, though they are widely used by nonbinary individuals 7 (Dev et al., 2021) (an exception is Munro and Morrison (2020), who write of “people who use ‘hers,’ ‘theirs’ and ‘themself’ to align their current…”
Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution (2022, background): “This difference in performance can lead to allocational harms when coreference resolution is used in down-stream tasks such as ranking authors based on citation counts in the bodies of texts (Dev et al., 2021).”
Measuring Harmful Sentence Completion in Language Models for LGBTQIA+ Individuals (2022, background): “Most recently, Dev et al. (2021) surveyed harms arising from gender-exclusivity in language technology.”
Theories of “Gender” in NLP Bias Research (2022, background): “6 [15] Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff M Phillips, and Kai-Wei Chang.”
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “, 2020), not inclusive in framing (Dev et al., 2021b), ambiguous about what bias is measured (Blodgett et al.”
“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation (2023, background): “In NLP, studies have explored misgendering with pronouns3 [3, 21], directed toxic language [43, 49], and the overfiltering content by and for queer individuals [27, 68].”
Back to the Future: On Potential Histories in NLP (2022, background): “NLP has long been ignoring neopronouns, leading to exclusion of these individuals in language technology (Cao and Daumé III, 2021; Dev et al., 2021).”
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers (2022, background): “Gender identity refers to the personal sense of one’s own gender [17, 39].”