Quantifying Social Biases in Contextual Word Representations

By Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W. Black, Yulia Tsvetkov. 2019

37 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “Pre-trained language models have been demonstrated to contain and amplify biases in underlying data (Sheng et al., 2021; Kurita et al., 2019; Dev et al., 2019).”
LLaMA: Open and Efficient Foundation Language Models (2023, background): “Large language models have been showed to reproduce and amplify biases that are existing in the training data (Sheng et al., 2019; Kurita et al., 2019), and to generate toxic or offensive content (Gehman et al.”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “Bi-ases may also be introduced in downstream tasks and representations in models where representations depend on additional context (Zhao et al., 2019b; Kurita et al., 2019).”
A Survey on Bias and Fairness in Natural Language Processing (2022, neutral): “Kurita et al. (2019) propose a new measure of bias for sentence representations which shows better correlation with human biases than SEAT and addresses some of the problems mentioned above.”
UNQOVERing Stereotypical Biases via Underspecified Questions (2020, background): “Several notable examples are coreference resolution (Rudinger et al., 2018; Zhao et al., 2018; Kurita et al., 2019), machine translation (Stanovsky et al., 2019; Cho et al., 2019), textual entailment (Dev et al., 2020a), language generation (Sheng et al., 2019), or clinical classiﬁ-cation (Zhang et…”
Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases (2020, methodology): “Prior methods use templates analogous to masked language modeling to measure bias in CWE [13, 14, 15].”
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2020, background): “It has since been translated into 6 other languages (XWEAT; Lauscher and Glavas, 2019), and extended to operate on full sentences (May et al., 2019) and on contextual language models (Kurita et al., 2019).”
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer (2020, background): “…in various components of the NLP systems, such as the training dataset (Zhao et al., 2018a; Rudinger et al., 2018), the embed-dings (Bolukbasi et al., 2016; Caliskan et al., 2017; Zhou et al., 2019; Manzini et al., 2019) as well as the pre-trained models (Zhao et al., 2019; Kurita et al., 2019).”