Perturbation Augmentation for Fairer NLP

By Rebecca Qian, Candace Ross, Jude Fernandes, Eric Michael Smith, Douwe Kiela, Adina Williams. 2022

73 citations recorded by the source corpus.

Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets. In this work, we ask whether training on demographically perturbed data leads to fairer language models. We collect a large dataset of human annotated text perturbations and train a neural perturbation model, which we show outperforms heuristic alternatives. We find that (i) language models (LMs) pre-trained on demographically perturbed corpora are typically more fair, and (ii) LMs finetuned on perturbed GLUE datasets exhibit less demographic bias on downstream tasks, and (iii) fairness improvements do not come at the expense of performance on downstream tasks. Lastly, we discuss outstanding questions about how best to evaluate the (un)fairness of large language models. We hope that this exploration of neural demographic perturbation will help drive more improvement towards fairer NLP.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models (2023, methodology): “This perturbation model is trained on Perturbation Augmentation NLP DAtaset (PANDA) (Qian et al., 2022) which is a human-generated dataset.”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, methodology): “, 2021), or use machine-learned systems to swap descriptors (Qian et al., 2022).”
Toxicity in Multilingual Machine Translation at Scale (2022, methodology): “This English-only (American English) dataset has been used to evaluate a variety of demographic biases in language modeling (Qian et al., 2022; Smith et al., 2022).”
Bias and Fairness in Large Language Models: A Survey (2023, background): “Counterfactual data augmentation (CDA) is one of the primary of these augmentation techniques (Lu et al., 2020; Qian et al., 2022; Webster et al., 2020; Zmigrod et al., 2019), replacing protected attribute words, such as gendered pronouns, to achieve a balanced dataset.”