Examining Gender Bias in Languages with Grammatical Gender

By Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, Kai-Wei Chang. 2019

115 citations recorded by the source corpus.

Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora. However, most studies to date have focused on quantifying and mitigating such bias only in English. These analyses cannot be directly extended to languages that exhibit morphological agreement on gender, such as Spanish and French. In this paper, we propose new metrics for evaluating gender bias in word embeddings of these languages and further demonstrate evidence of gender bias in bilingual embeddings which align these languages with English. Finally, we extend an existing approach to mitigate gender bias in word embedding of these languages under both monolingual and bilingual settings. Experiments on modified Word Embedding Association Test, word similarity, word translation, and word pair translation tasks show that the proposed approaches can effectively reduce the gender bias while preserving the utility of the original embeddings.

Subfields: Artificial Intelligence, Gender Studies, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Gender Studies

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…Maudslay et al. (2019); May et al. (2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao et al. (2019); Zhou et al. (2019); Ethayarajh (2020); Huang et al. (2020); Jia et al. (2020); Popović et al. (2020); Pryzant et al. (2020); Vig et al. (2020); Wang…”
The Woman Worked as a Babysitter: On Biases in Language Generation (2019, background): “…these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation contexts.”
Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation (2020, background): “This follows precedent set in Johnson et al. (2017) and Zhou et al. (2019) for controlling translations via tag-based annotations.”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “This word embedding work has extended to multilingual work on gender-marking (Gonen et al., 2019; Williams et al., 2019; Zhou et al., 2019).”
Multi-Dimensional Gender Bias Classification (2020, background): “, 2020)—including multilingual ones (Escudé Font and Costa-jussà, 2019; Gonen et al., 2019; Zhou et al., 2019)—and affect a wide range of downstream tasks includ-”
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, background): “…resolution (Zhao et al., 2018a), named entity recognition (Mehrabi et al., 2020), machine translation (Font and Costa-jussà, 2019), word embedding (Bolukbasi et al., 2016; Zhao et al., 2018b; Zhou et al., 2019), as well as surveys (Sun et al., 2019; Blodgett et al., 2020; Mehrabi et al., 2021).”
Towards Controllable Biases in Language Generation (2020, background): “., 2018b), debiasing contextualized embeddings (Zhao et al., 2019; Kurita et al., 2019), extending bias analyses to multiple demographic groups (Manzini et al., 2019), and debiasing across languages (Zhou et al., 2019; Zmigrod et al., 2019)). Much of the work on bias analysis and mitigation in natur”
Towards Debiasing Sentence Representations (2020, background): “We acknowledge that biases can manifest differently across different languages, in particular gendered languages (Zhou et al., 2019), and emphasize the need for future extensions in these directions.”
HONEST: Measuring Hurtful Sentence Completion in Language Models (2021, background): “Only McCurdy and Serbetçi (2017); Zhou et al. (2019) examine the bias in word embeddings of gender-inflected languages, demonstrating the need for an adequate framework different from the ones proposed for English.”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “For instance, many gender bias evaluations focus on Western professions, which are grammatically gendered in some languages (Chen et al., 2021; Zhou et al., 2019) or may not cover other prevalent occupations outside the U.S. (Escudé Font and Costa-jussà, 2019).”
On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs (2020, background): “Finally, many scholars have proposed a variety of computational techniques for mitigating gender norms and stereotypes in a wide range of languagebased applications (Dev and Phillips, 2019; Dinan et al., 2019; Ethayarajh et al., 2019; Hall Maudslay et al., 2019; Stanovsky et al., 2019; Tan and Celis”
Socially Aware Bias Measurements for Hindi Language Representations (2021, unknown): “Some recent work tackle the challenges of societal biases in language representations coming from various cultures and languages such as Arabic, French, Spanish and German (Lauscher et al., 2020; Chávez Mulsa and Spanakis, 2020; Kurpicz-Briki, 2020; Zhou et al., 2019).”
Theories of “Gender” in NLP Bias Research (2022, unknown): “18653/v1/D18-1521 [174] Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, and Kai-Wei Chang.”
AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings (2020, background): “Zhou et al. (2019) focus on measuring bias in languages with grammatical gender.”
Gender Bias in Machine Translation (2021, background): “However, their survey is based on monolingual applications, whose underlying assumptions and solutions may not be directly applicable to languages other than English (Zhou et al., 2019; Zhao et al., 2020; Takeshita et al., 2020) and cross-lingual settings.”
Efficient Gender Debiasing of Pre-trained Indic Language Models (2022, background): “Zhou et al. (2019) was one of the ﬁrst papers to quantify bias in gendered languages like Spanish and French.”
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer (2020, methodology): “…in various components of the NLP systems, such as the training dataset (Zhao et al., 2018a; Rudinger et al., 2018), the embed-dings (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhou et al., 2019; Manzini et al., 2019) as well as the pre-trained models (Zhao et al., 2019; Kurita et al., 2019).”
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2020, result): “It is important to take a language with pervasive gender-marking (Spanish) into account, as previous work has shown that grammatical gender-marking has a strong effect on gender bias (McCurdy and Serbetci, 2017; Gonen et al., 2019; Zhou et al., 2019).”