Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer

By Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, Ahmed Hassan Awadallah. 2020

97 citations recorded by the source corpus.

Multilingual representations embed words from many languages into a single semantic space such that words with similar meanings are close to each other regardless of the language. These embeddings have been widely used in various settings, such as cross-lingual transfer, where a natural language processing (NLP) model trained on one language is deployed to another language. While the cross-lingual transfer techniques are powerful, they carry gender bias from the source to target languages. In this paper, we study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications. We create a multilingual dataset for bias analysis and propose several ways for quantifying bias in multilingual representations from both the intrinsic and extrinsic perspectives. Experimental results show that the magnitude of bias in the multilingual representations changes differently when we align the embeddings to different target spaces and that the alignment direction can also have an influence on the bias in transfer learning. We further provide recommendations for using the multilingual word representations for downstream tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…(2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao et al. (2019); Zhou et al. (2019); Ethayarajh (2020); Huang et al. (2020); Jia et al. (2020); Popović et al. (2020); Pryzant et al. (2020); Vig et al. (2020); Wang et al. (2020); Zhao et al. (2020).”
UNQOVERing Stereotypical Biases via Underspecified Questions (2020, methodology): “…models, often via similarity-based analysis of the biases in input representations (Bolukbasi et al., 2016a; Garg et al., 2018; Chaloner and Maldonado, 2019; Bordia and Bow-man, 2019; Tan and Celis, 2019; Zhao et al., 2019, 2020), or an intermediate classiﬁcation task (Re-casens et al., 2013).”
Theories of “Gender” in NLP Bias Research (2022, background): “[168] Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Hassan Awadallah.”
Few-shot Learning with Multilingual Language Models (2021, methodology): “We follow the setup in (Zhao et al., 2020) where people’s names and pronouns are removed from the bios.”
Gender Bias in Machine Translation (2021, background): “However, their survey is based on monolingual applications, whose underlying assumptions and solutions may not be directly applicable to languages other than English (Zhou et al., 2019; Zhao et al., 2020; Takeshita et al., 2020) and cross-lingual settings.”
Can Existing Methods Debias Languages Other than English? First Attempt to Analyze and Mitigate Japanese Word Embeddings (2020, background): “Spanish and French, and how they differ depending on language (Zhou et al., 2019; Zhao et al., 2020).”
Benchmarking Intersectional Biases in NLP (2022, background): “For instance, all four of the aforementioned allocational harm papers measure and/or mitigate gender bias with respect to an NLP-based occupation classifier (De-Arteaga et al., 2019; Prost et al., 2019; Romanov et al., 2019; Zhao et al., 2020).”
A Survey on Gender Bias in Natural Language Processing (2021, methodology): “2020] and Spanish [Zhao et al. 2020] with the notable exception of a broader multilingual analysis of gender bias in machine translation [Prates et al.”
On Measures of Biases and Harms in NLP (2021, background): “Zhao et al. (2020) extend the original dataset to Spanish, French, and German.”
Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Models (2021, methodology): “It has been adopted as an extrinsic measure by works such as Webster et al. (2020) and Zhao et al. (2020).”
Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold (2022, background): “Zhao et al. (2020) study gender bias in multilingual word embeddings and cross-lingual transfer.”
COLD: A Benchmark for Chinese Offensive Language Detection (2022, background): “Due to the culture differences, the alignment from English to Chinese is a challenging task [Ahmad et al., 2019; Zhao et al., 2020], hence limits the performance of crosslanguage methods [Su et al.”
French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English (2022, background): “Zhao et al. (2020) study gender bias in a multilingual context.”
Gender-Aware Reinflection using Linguistically Enhanced Neural Models (2020, background): “Zhao et al. (2020) studied gender bias which is exhibited by multilingual embeddings in four languages (English, German, French, and Spanish) and demonstrated that such bias can impact cross-lingual transfer learning tasks.”
What do Bias Measures Measure? (2021, background): “Zhao et al. (2020) extend the original dataset to Spanish, French, and German.”
Socially Aware Bias Measurements for Hindi Language Representations (2021, background): “Some recent work tackle the challenges of societal biases in language representations coming from various cultures and languages such as Arabic, French, Spanish and German (Lauscher et al., 2020; Chávez Mulsa and Spanakis, 2020; Kurpicz-Briki, 2020; Zhou et al., 2019).”