“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation

By Anaelia Ovalle, Palash Goyal, Jwala Dhamala, Zachary Jaggers, Kai-Wei Chang, Aram Galstyan, Richard S. Zemel, Rahul Gupta. 2023

99 citations recorded by the source corpus.

Warning: This paper contains examples of gender non-affirmative language which could be offensive, upsetting, and/or triggering. Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life. Given the recent popularity and adoption of language generation technologies, the potential to further marginalize this population only grows. Although a multitude of NLP fairness literature focuses on illuminating and addressing gender biases, assessing gender harms for TGNB identities requires understanding how such identities uniquely interact with societal gender norms and how they differ from gender binary-centric perspectives. Such measurement frameworks inherently require centering TGNB voices to help guide the alignment between gender-inclusive NLP and whom they are intended to serve. Towards this goal, we ground our work in the TGNB community and existing interdisciplinary literature to assess how the social reality surrounding experienced marginalization of TGNB persons contributes to and persists within Open Language Generation (OLG). This social knowledge serves as a guide for evaluating popular large language models (LLMs) on two key aspects: (1) misgendering and (2) harmful responses to gender disclosure. To do this, we introduce TANGO, a dataset of template-based real-world text curated from a TGNB-oriented community. We discover a dominance of binary gender norms reflected by the models; LLMs least misgendered subjects in generated text when triggered by prompts whose subjects used binary pronouns. Meanwhile, misgendering was most prevalent when triggering generation with singular they and neopronouns. When prompted with gender disclosures, TGNB disclosure generated the most stigmatizing language and scored most toxic, on average. Our findings warrant further research on how TGNB harms manifest in LLMs and serve as a broader case study toward concretely grounding the design of gender-inclusive AI in community voices and interdisciplinary literature.

Subfields: Artificial Intelligence, Communication, Gender Studies, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

In Gender Studies

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Human-led evaluations [173] for hateful and sensitive content can have a high psychological cost, as seen in content moderation labor (see 4.1.7 Data and Content Moderation Labor).”
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters (2023, background): “…that Natural Language Generation (NLG) models are gender biased (Sheng et al., 2019, 2020; Dinan et al., 2020; Sheng et al., 2021a; Bender et al., 2021) and therefore pose a risk to harm minorities when used in sensitive applications (Sheng et al., 2021b; Ovalle et al., 2023a; Prates et al., 2018).”