On Measuring and Mitigating Biased Inferences of Word Embeddings

By Sunipa Dev, Tao Li, Jeff M. Phillips, Vivek Srikumar. 2019

198 citations recorded by the source corpus.

Word embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them. We use this observation to design a mechanism for measuring stereotypes using the task of natural language inference. We demonstrate a reduction in invalid inferences via bias mitigation strategies on static word embeddings (GloVe). Further, we show that for gender bias, these techniques extend to contextualized embeddings when applied selectively only to the static components of contextualized embeddings (ELMo, BERT).

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “Pre-trained language models have been demonstrated to contain and amplify biases in underlying data (Sheng et al., 2021; Kurita et al., 2019; Dev et al., 2019).”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…(2018); Zhang et al. (2018); Badjatiya et al. (2019); Bhargava and Forsyth (2019); Cao and Daumé (2019); Cho et al. (2019); Davidson et al. (2019); Dev et al. (2019); Garimella et al. (2019); Garg et al. (2019); Huang et al. (2019); James-Sorenson and AlvarezMelis (2019); Kaneko and Bollegala…”
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021, neutral): “A bulk of social bias studies on language models have focused on binary gender and the stereotypes associated with masculine and feminine attributes (Bolukbasi et al., 2016; Webster et al., 2018; Dev et al., 2020b).”
Galactica: A Large Language Model for Science (2022, background): “Models may mirror misinformation, stereotypes and bias in the corpus (Sheng et al., 2019; Kurita et al., 2019; Dev et al., 2019; Blodgett et al., 2020; Sheng et al., 2021).”
Re-contextualizing Fairness in NLP: The Case of India (2022, background): “Religion: Religious biases have been studied in NLP (Dev et al., 2020; Nadeem et al., 2020; Abid et al., 2021), however the social disparities and stereotypes about various religious groups differ significantly in India from the West, (Malik et al., 2021).”
BBQ: A hand-built bias benchmark for question answering (2021, background): “LMs have been found to reproduce social biases in downstream tasks, such as textual entailment (Dev et al., 2020; Sharma et al., 2021), language generation (Sheng et al., 2019), and coreference resolution (Rudinger et al., 2018).”
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, neutral): “Our work also touches upon the fairness literature (e.g., Bolukbasi et al., 2016; Dev et al., 2020; Chang et al., 2019; Blodgett et al., 2020; Sun et al., 2019).”
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP (2021, methodology): “…illustrating that both static (e.g., Mikolov et al., 2013; Bojanowski et al., 2017) and contextualized word embeddings (e.g., Peters et al., 2018; Devlin et al., 2019) pretrained in a self-supervised fashion exhibit all kinds of unfair and discriminative biases (Bolukbasi et al., 2016; Caliskan…”
UNQOVERing Stereotypical Biases via Underspecified Questions (2020, methodology): “These biases are subsequently passed on to downstream tasks such as co-reference resolution (Rudinger et al., 2018; Zhao et al., 2018), textual entailment (Dev et al., 2020a), and translation (Stanovsky et al., 2019).”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “Although word embeddings may help identify biases in the context of LLMs, it is often difﬁcult to access the learned contextual language representations of the model (Abid et al., 2021; Dev et al., 2020).”
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021, background): “, 2019) and natural language inference (Dev et al., 2020).”
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “Dev et al. (2020)’s measure based on NLI (BIASNLI) evaluates if stereotypical inferences are made by language models.”
On Measures of Biases and Harms in NLP (2021, background): “Dev et al. (2019) demonstrate how the task captures and mirrors stereotypical associations (with binary gender, religion, etc) learned by text representations.”
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model (2022, background): “Previous work has shown that large-scale pre-trained language models (LMs) can demonstrate undesirable biases (Sheng et al., 2021; Kurita et al.; Dev et al., 2020; Liang et al., 2021; Weidinger et al., 2022).”
Bias and Fairness in Large Language Models: A Survey (2023, methodology): “Bias NLI (Dev et al., 2020) alternatively probes for bias through inference tasks.”
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2020, methodology): “For measuring bias, we use existing intrinsic [3, 6] and extrinsic measures [5].”
Measuring Harmful Sentence Completion in Language Models for LGBTQIA+ Individuals (2022, background): “…2016; Gonen and Goldberg, 2019; Lauscher et al., 2020; Bianchi et al., 2021), and unfairly biased task-specific models, e.g., for natural language inference (e.g., Dev et al., 2020), natural language generation (e.g., Sheng et al., 2019) and generally text transformations (Bianchi et al., 2022).”