Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

By Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang. 2018

1,177 citations recorded by the source corpus.

In this paper, we introduce a new benchmark for co-reference resolution focused on gender bias, WinoBias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing datasets.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…developed for a breadth of tasks including language modeling (Lu et al., 2018; Bordia and Bowman, 2019), coreference resolution (Rudinger et al., 2018; Zhao et al., 2018a), machine translation (Vanmassenhove et al., 2018; Stanovsky et al., 2019), sentiment analysis (Kiritchenko and Mohammad,…”
StereoSet: Measuring stereotypical bias in pretrained language models (2020, methodology): “Another popular method to evaluate bias of pre-trained representations is to measure bias on extrinsic applications like coreference resolution (Rudinger et al., 2018; Zhao et al., 2018) and sentiment analysis (Kiritchenko and Mohammad, 2018).”
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, background): “Systematic imbalances were observed across NLP tasks, such as gender bias in coreference resolution (Zhao et al., 2018; Rudinger et al., 2018), visual semantic role labeling (Zhao et al.”
The Woman Worked as a Babysitter: On Biases in Language Generation (2019, methodology): “…analyze these two bias contexts because the occupation context has been well-studied in other tasks (Bolukbasi et al., 2016; Rudinger et al., 2018; Zhao et al., 2018; Zhou et al., 2019), and the more descriptive language in respect contexts are a good contrast for the more subtle occupation…”
WinoGrande (2019, methodology): “WinoBias (Zhao et al., 2018) This is a concurrent work with Winogender, aimed at diagnosing gender bias in coreference resolution systems.”
Gender Bias in Coreference Resolution (2018, methodology): “…with extensive analysis of observed system bias, revealing its correlation with biases present in real-world and textual statistics; by contrast, Zhao et al. (2018) present methods of debi-asing existing systems, showing that simple approaches such as augmenting training data with…”
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts (2021, background): “…present in written text collections including correlations between gender and profession (Bolukbasi et al., 2016; Rudinger et al., 2018b; Zhao et al., 2018), negative sentiment about different racial and religious groups (Li et al., 2020; Nadeem et al., 2021), and persons with…”
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021, background): “ten rely on gendered information for decision making, such as in named entity recognition, coreference resolution, and machine translation (Mehrabi et al., 2020; Zhao et al., 2018; Stanovsky et al., 2019), but the purview of gender in these tasks and”
Toward Gender-Inclusive Coreference Resolution (2019, background): “In comparison to [Zhao et al. 2018] and [Rudinger et al. 2018] (as well as in contrast to our MAP dataset), we focused on naturally occurring data, but sampled specifically to surface more gender-related phenomena than may be found in, say, the Wall Street Journal.”
Evaluating Gender Bias in Machine Translation (2019, methodology): “Following these observations, we design a challenge set approach for evaluating gender bias in MT using a concatenation of Winogender and WinoBias.”
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets (2021, background): “These datasets consist of tests that can take a variety of formats, including sentence templates where terms pertaining to groups or their attributes are perturbed (Rudinger et al., 2018; Zhao et al., 2018; Kurita et al., 2019), prompts designed to elicit problematic responses (Gehman et al.”
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, background): “For example, pro and anti versions of type 1 and type 2 Winobias (Zhao et al., 2018) tasks from PromptSource, and all 57 subtasks of MMLU (Hendrycks et al., 2020), would be aggregated to get per task performance.”
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020, background): “These language models, and embeddings extracted from them, have been shown to ∗Equal contribution. learn and use these biases (Bolukbasi et al., 2016; Caliskan et al., 2017; Garg et al., 2017; May et al., 2010; Zhao et al., 2018; Rudinger et al., 2017).”
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models (2023, methodology): “The first stage is conceptualization and clarifying what entity is being measured.”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “…about the gender of a particular individual: for example, if you know someone works a secretary in the U.S., then you might reasonably infer that person is likely to identify as a woman, because 95% of surveyed secretaries did so according to the U.S. Department of Labor (Zhao et al., 2018).”
BBQ: A hand-built bias benchmark for question answering (2021, background): “, 2020) or gender-occupation associations (Rudinger et al., 2018; Zhao et al., 2018).”