On Measures of Biases and Harms in NLP

By Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, M. Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng and 1 more. 2021

126 citations recorded by the source corpus.

Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. To create interventions and mitigate these biases and associated harms, it is vital to be able to detect and measure such biases. While existing works propose bias evaluation and mitigation methods for various tasks, there remains a need to cohesively understand the biases and the specific harms they measure, and how different measures compare with each other. To address this gap, this work presents a practical framework of harms and a series of questions that practitioners can answer to guide the development of bias measures. As a validation of our framework and documentation questions, we also present several case studies of how existing bias measures in NLP -- both intrinsic measures of bias in representations and extrinsic measures of bias of downstream applications -- can be aligned with different harms and how our proposed documentation questions facilitates more holistic understanding of what bias measures are measuring.

Subfields: Artificial Intelligence, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In AI Ethics, Fairness & Society

Kai-Wei Chang

Citations in the corpus

Undesirable biases in NLP: Averting a crisis of measurement (2022, methodology): “The existence of subconcepts of model bias has already been hinted at by some researchers (e.g., Du et al., 2021; Dev et al., 2022) and breaking down the bias construct into subcomponents and devising subtests for them comes with practical advantages: It is difficult to define “model bias” and a…”
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters (2023, methodology): “Researchers have studied methods to evaluate and mitigate the two types of biases in Natural Language Understanding (NLU) (Bolukbasi et al., 2016; Dev et al., 2022; Dixon et al., 2018; Bordia and Bowman, 2019; Zhao et al., 2017, 2018; Sun and Peng, 2021) and Natural Language Generation (NLG) tasks (”