A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings

By Chris Sweeney, Maryam Najafian. 2019

62 citations recorded by the source corpus.

Word embedding models have gained a lot of traction in the Natural Language Processing community, however, they suffer from unintended demographic biases. Most approaches to evaluate these biases rely on vector space based metrics like the Word Embedding Association Test (WEAT). While these approaches offer great geometric insights into unintended biases in the embedding vector space, they fail to offer an interpretable meaning for how the embeddings could cause discrimination in downstream NLP applications. In this work, we present a transparent framework and metric for evaluating discrimination across protected groups with respect to their word embedding bias. Our metric (Relative Negative Sentiment Bias, RNSB) measures fairness in word embeddings via the relative negative sentiment associated with demographic identity terms from various protected groups. We show that our framework and metric enable useful analysis into the bias in word embeddings.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…Nozza et al. (2019); Prabhakaran et al. (2019); Romanov et al. (2019); Ruane et al. (2019); Sap et al. (2019); Sheng et al. (2019); Sun et al. (2019); Sweeney and Najafian (2019); Vaidya et al. (2019); Gaut et al. (2020); Gencoglu (2020); Hovy et al. (2020); Hutchinson et al. (2020); Kim et al.…”
On Measuring and Mitigating Biased Inferences of Word Embeddings (2019, background): “To our knowledge, this is the among the ﬁrst demonstrations (Sweeney and Najaﬁan 2019; Manzini et al. 2019) of national or religious bias in word embeddings.”
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, background): “Although Sweeney and Najaﬁan (2019) have previously shown that ConceptNet word embed-dings (Speer, 2017) are less biased compared to other embeddings, we demonstrate that destructive biases still exist in ConceptNet that need to be carefully studied.”
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus (2021, background): “Consequently, much work has also been dedicated to creating standardized diagnostic tests to detect these biases (Caliskan et al., 2017; May et al., 2019; Nadeem et al., 2020; Sweeney and Najafian, 2019) and to remove them (Bolukbasi et al.”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “This mirrors predictive bias in standardized testing (Swinton, 1981): “a [predictive model] cannot be called biased without reference to a specific prediction situation; thus, the same instrument may be biased in one application, but unbiased in another."”
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability (2020, background): “To our knowledge, this is the among the first demonstrations [62, 90] of national or religious bias in word embeddings.”
Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning (2020, background): “Starting with word embeddings, works like [5, 6, 10, 24] have made progress towards identifying and mitigating many different types of bias with respect to gender and race.”
Evaluating Gender Bias in Hindi-English Machine Translation (2021, background): “We calculate the RNSB by deﬁning the Kulback-Leibler divergence of P from U to assess the similarity of these distributions.”