What are the Biases in My Word Embedding?

By Nathaniel Swinger, Maria De-Arteaga, IV NeilThomasHeffernan, Mark D. M. Leiserson, Adam Tauman Kalai. 2018

107 citations recorded by the source corpus.

This paper presents an algorithm for enumerating biases in word embeddings. The algorithm exposes a large number of offensive associations related to sensitive features such as race and gender on publicly available embeddings, including a supposedly "debiased" embedding. These biases are concerning in light of the widespread use of word embeddings. The associations are identified by geometric patterns in word embeddings that run parallel between people's names and common lower-case tokens. The algorithm is highly unsupervised: it does not even require the sensitive features to be pre-specified. This is desirable because: (a) many forms of discrimination?such as racial discrimination-are linked to social constructs that may vary depending on the context, rather than to categories with fixed definitions; and (b) it makes it easier to identify biases against intersectional groups, which depend on combinations of sensitive features. The inputs to our algorithm are a list of target tokens, e.g. names, and a word embedding. It outputs a number of Word Embedding Association Tests (WEATs) that capture various biases present in the data. We illustrate the utility of our approach on publicly available word embeddings and lists of names, and evaluate its output using crowdsourcing. We also show how removing names may not remove potential proxy bias.

Subfields: Artificial Intelligence, Sociology and Political Science, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…et al. (2019); Liang et al. (2019); Maudslay et al. (2019); May et al. (2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao et al. (2019); Zhou et al. (2019); Ethayarajh (2020); Huang et al. (2020); Jia et al. (2020); Popović et al. (2020);…”
Towards Understanding and Mitigating Social Biases in Language Models (2021, background): “However, recent work has shown that discriminative models including pretrained word and sentence embeddings reﬂect and propagate social biases present in training corpora (Bolukbasi et al., 2016; Caliskan et al., 2017; Lauscher and Glavaˇs, 2019; Swinger et al., 2019).”
Measuring Bias in Contextualized Word Representations (2019, background): “, 2017; Yao and Huang, 2017). There have been several efforts to investigate the amount of intrinsic bias within uncontextualized word embeddings in binary (Bolukbasi et al., 2016; Garg et al., 2018; Swinger et al., 2019) and multiclass (Manzini et al., 2019) settings. Contextualized embeddings such”
CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, neutral): “Comparatively, existing measures model bias using observational, rather than interventional measures (Rudinger, May, and Durme 2017; DeArteaga et al. 2019; Davidson, Bhattacharya, and Weber 2019; Swinger et al. 2019; Ravfogel et al. 2020).”
Towards Debiasing Sentence Representations (2020, background): “Although word embeddings [19, 16] are useful for a variety of tasks, recent work has shown that they reflect and propagate social biases present in training corpora [12, 6, 25, 5].”
What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes (2019, background): “In particular, we build on the work of Swinger et al. (2019), which showed that word embeddings of names typically reﬂect the societal biases that are associated with those names, including race, gender, and age biases, as well encoding information about other factors that inﬂuence naming practices…”
Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases (2020, methodology): “[60] presented an adaptation of the WEAT to identify biases associated with clusters of names.”
Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types (2020, methodology): “One of the additional lexicons tested, the WEAT lexicon, deserves special consideration since previous works have used this small size lexicon (N = 50) when testing for bias in word embedding models [8,15,16].”
Can Existing Methods Debias Languages Other than English? First Attempt to Analyze and Mitigate Japanese Word Embeddings (2020, methodology): “After that, the most relevant words were chosen as t in each Voronoi set, and we set t = 3 , following Swinger et al. (2019).”
Hindi-English Hate Speech Detection: Author Profiling, Debiasing, and Practical Perspectives (2020, methodology): “(Swinger et al. 2018) proposed an Unsupervised Bias Enumeration (UBE) algorithm, which is used to enumerate sets of bias-inducing words in an unsupervised manner.”
Debiasing Gender biased Hindi Words with Word-embedding (2019, background): “Recently Swinger et al. [11] show that word embeddings of names typically reflect the societal biases that are associated with those names, including race, gender, and age biases.”
Quantifying Social Biases in Contextual Word Representations (2019, background): “There have been several efforts to investigate the amount of intrinsic bias within uncontextualized word embeddings in binary (Bolukbasi et al., 2016; Garg et al., 2018; Swinger et al., 2019) and multiclass (Manzini et al., 2019) settings.”
Theories of “Gender” in NLP Bias Research (2022, background): “00461 [50] Joel Escudé Font and Marta R Costa-Jussà.”
Leveraging Bias in Pre-Trained Word Embeddings for Unsupervised Microaggression Detection (2021, unknown): “As such, they often exhibit gender and racial bias (Swinger et al., 2019).”
Cross-Loss Influence Functions to Explain Deep Network Representations (2020, background): “Bolukbasi et al. (2016) were among the ﬁrst to discover the inherent biases in word embeddings, a ﬁnding that has been replicated in transformers (Basta et al., 2019; Kurita et al., 2019; Silva et al., 2021) and more (Costa-jussà et al., 2019; Swinger et al., 2019; Nissim et al., 2020).”