Bias in word embeddings

By Orestis Papakyriakopoulos, Simon Hegelich, J. M. Serrano, F. Marco. 2020

109 citations recorded by the source corpus.

Word embeddings are a widely used set of natural language processing techniques that map words to vectors of real numbers. These vectors are used to improve the quality of generative and predictive models. Recent studies demonstrate that word embeddings contain and amplify biases present in data, such as stereotypes and prejudice. In this study, we provide a complete overview of bias in word embeddings. We develop a new technique for bias detection for gendered languages and use it to compare bias in embeddings trained on Wikipedia and on political social media data. We investigate bias diffusion and prove that existing biases are transferred to further machine learning models. We test two techniques for bias mitigation and show that the generally proposed methodology for debiasing models at the embeddings level is insufficient. Finally, we employ biased word embeddings and illustrate that they can be used for the detection of similar biases in new data. Given that word embeddings are widely used by commercial companies, we discuss the challenges and required actions towards fair algorithmic implementations and applications.

Subfields: Artificial Intelligence, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…Webster et al. (2019); Zmigrod et al. (2019); Gyamfi et al. (2020); Hube et al. (2020); Hutchinson et al. (2020); Kim et al. (2020); Nadeem et al. (2020); Papakyriakopoulos et al. (2020); Ravfogel et al. (2020); Rozado (2020); Sen and Ganguly (2020); Shin et al. (2020); Strengers et al. (2020).”
Censorship of Online Encyclopedias: Implications for NLP Models (2021, background): “While these word associations can be of interest to social science researchers, they may cause harm if used in downstream tasks [3, 29].”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, unknown): “…(Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldon-ado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2020; Liang et al., 2020; Papakyriakopoulos et al., 2020).”
Socio-Technological Challenges and Opportunities: Paths Forward (2021, background): “…et al., 2017, Whittaker et al., 2018, Speicher et al., 2018, Chouldechova et al., 2018, Ekstrand et al., 2018, Selbst et al., 2019, Ali et al., 2019, Babaei et al., 2019, Crawford et al., 2019, Papakyriakopoulos et al., 2020, Park et al., 2021, Bender et al., 2021, Kleinberg and Raghavan, 2021].”
Theories of “Gender” in NLP Bias Research (2022, background): “[105] Orestis Papakyriakopoulos, Simon Hegelich, Juan Carlos, Medina Serrano, and Fabienne Marco.”
Probing Toxic Content in Large Pre-Trained Language Models (2021, background): “Similarly to how long existing stereotypes are deep-rooted in word embeddings (Papakyriakopoulos et al., 2020; Garg et al., 2018), PTLMs have also been shown to recreate stereotypical content due to the nature of their training data (Sheng et al.”
Open-Source Embedding Models: A Comprehensive Survey of Techniques, Benchmarks, and Applications (2026, unknown): “Embedding models are trained on vast amounts of text data, and if this data contains societal biases related to gender, race, or other demographic attributes, these biases can be inadvertently encoded into the vector representations learned by the model [82].”