An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models

By Nicholas Meade, Elinor Poole-Dayan, Siva Reddy. 2021

165 citations recorded by the source corpus.

Recent work has shown pre-trained language models capture social biases from the large amounts of text they are trained on. This has attracted attention to developing techniques that mitigate such biases. In this work, we perform an empirical survey of five recently proposed bias mitigation techniques: Counterfactual Data Augmentation (CDA), Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebias. We quantify the effectiveness of each technique using three intrinsic bias benchmarks while also measuring the impact of these techniques on a model’s language modeling ability, as well as its performance on downstream NLU tasks. We experimentally find that: (1) Self-Debias is the strongest debiasing technique, obtaining improved scores on all bias benchmarks; (2) Current debiasing techniques perform less consistently when mitigating non-gender biases; And (3) improvements on bias benchmarks such as StereoSet and CrowS-Pairs by using debiasing strategies are often accompanied by a decrease in language modeling ability, making it difficult to determine whether the bias mitigation was effective.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Alignment & RLHF

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Alignment & RLHF

Citations in the corpus

Using In-Context Learning to Improve Dialogue Safety (2023, background): “Meade et al. (2022) empirically demonstrated Self-Debias can be used to mitigate multiple social biases during generation.”
Perturbation Augmentation for Fairer NLP (2022, background): “, 2021), which has also been shown to be an effective approach (Meade et al., 2022), can be combined with our approach to even greater effect.”
Efficient Gender Debiasing of Pre-trained Indic Language Models (2022, background): “Meade et al. (2022) surveys various mitigation strategies used to reduce gender bias in pre-trained language models like dropout, ﬁne-tuning, sentence debias, and iterative null spacing.”
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2022, background): “A specific failure involves stereotype bias—a range of benchmarks have outlined a general pattern of stereotypical behaviour in language models (Meade et al., 2022; Nadeem et al., 2021; Nangia et al., 2020; Parrish et al., 2022).”
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey (2023, methodology): “A short survey of pre-trained language models for conversational ai-a new age in nlp [26] 2020 ACSWM NLP DC, 4 A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models [27] 2022 arXiv NLP SC, 34 Vision-Language Intelligence: Tasks, Representation Learning, and Lar”
Bilingual language model for protein sequence and structure (2024, unknown): “If so, ProstT5 might have picked up such bias as common to any other LLM that may amplify bias in its training data [62].”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “The study first assessed the question-answering performance of these LLMs on Cisco certifications with varying difficulty levels, then examined their abilities in solving CTF challenges.”