Evaluating Gender Bias in Natural Language Inference

By Shanya Sharma Sharma, Manan Dey, Koustuv Sinha. 2021

46 citations recorded by the source corpus.

Gender-bias stereotypes have recently raised significant ethical concerns in natural language processing. However, progress in detection and evaluation of gender bias in natural language understanding through inference is limited and requires further investigation. In this work, we propose an evaluation methodology to measure these biases by constructing a challenge task that involves pairing gender-neutral premises against a gender-specific hypothesis. We use our challenge task to investigate state-of-the-art NLI models on the presence of gender stereotypes using occupations. Our findings suggest that three models (BERT, RoBERTa, BART) trained on MNLI and SNLI datasets are significantly prone to gender-induced prediction errors. We also find that debiasing techniques such as augmenting the training dataset to ensure a gender-balanced dataset can help reduce such bias in certain cases.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

BBQ: A hand-built bias benchmark for question answering (2021, background): “LMs have been found to reproduce social biases in downstream tasks, such as textual entailment (Dev et al., 2020; Sharma et al., 2021), language generation (Sheng et al., 2019), and coreference resolution (Rudinger et al., 2018).”
Memory-assisted prompt editing to improve GPT-3 after deployment (2022, background): “Finally, large-language models generate text that might be biased and insensitive to a user’s socio-cultural context (Bordia and Bowman, 2019; Sharma et al., 2021; Hovy and Prabhumoye, 2021).”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, methodology): “The majority of previous work on bias has focused par-694 ticularly on gender bias (Zhao et al., 2018; Stanovsky 695 et al., 2019; Levy et al., 2021; Sharma et al., 2021) and the very few works (Nadeem et al., 2020; Nangia et al., 2020) that take other dimensions of biases into account, have their…”
Theories of “Gender” in NLP Bias Research (2022, background): “[132] Shanya Sharma, Manan Dey, and Koustuv Sinha.”