Undesirable biases in NLP: Averting a crisis of measurement

By Oskar van der Wal, Dominik Bachmann, Alina Leidinger, Leendert van Maanen, Willem H. Zuidema, Katrin Schulz. 2022

8 citations recorded by the source corpus.

As Large Language Models and Natural Language Processing (NLP) technology rapidly develops and spreads into daily life, it becomes crucial to anticipate how its use could harm people. One problem that has received a lot of attention in recent years is that this technology has displayed harmful biases in its behavior. Although a lot of effort has been invested in assessing and mitigating these biases, our methods of measuring the biases of NLP models have serious problems (e.g., it is often unclear what they actually measure). In this paper, we provide an interdisciplinary approach to discussing the issue of NLP model bias by adopting the lens of psychometrics -- a field specialized in the measurement of concepts like bias that are not directly observable. In particular, we will explore two central notions from psychometrics, the construct validity and the reliability of measurement tools, and discuss how they can be applied in the context of measuring model bias. Our goal is to provide NLP practitioners with methodological tools for designing better bias measures, and to inspire them more generally to explore tools from psychometrics when working on bias measurement tools.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Stella Biderman
2. Oskar van der Wal
3. Hailey Schoelkopf

In Safety, Robustness & Interpretability

1. Stella Biderman
2. Oskar van der Wal
3. Edward Raff

Citations in the corpus

• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, methodology): “…Kirtane et al., 2022; Choenni et al., 2021), or the relationship between the corpus statistics and the measured bias (Bordia & Bowman, 2019; Van der Wal et al., 2022b), researchers have generally lacked the tools to study the role of the training data on the learning dynamics of bias in large…”