This Prompt is Measuring : Evaluating Bias Evaluation in Language Models

By Seraphina Goldfarb-Tarrant, Eddie L. Ungless, Esma Balkir, Su Lin Blodgett. 2023

17 citations recorded by the source corpus.

Bias research in NLP seeks to analyse models for social biases, thus helping NLP practitioners uncover, measure, and mitigate social harms. We analyse the body of work that uses prompts and templates to assess bias in language models. We draw on a measurement modelling framework to create a taxonomy of attributes that capture what a bias test aims to measure and how that measurement is carried out. By applying this taxonomy to 90 bias tests, we illustrate qualitatively and quantitatively that core aspects of bias test conceptualisations and operationalisations are frequently unstated or ambiguous, carry implicit assumptions, or be mismatched. Our analysis illuminates the scope of possible bias types the field is able to measure, and reveals types that are as yet under-researched. We offer guidance to enable the community to explore a wider section of the possible bias space, and to better close the gap between desired outcomes and experimental design, both for bias and for evaluating language models more broadly.

Subfields: Artificial Intelligence, Large Language Models, Evaluation & Benchmarks, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Evaluation & Benchmarks

In AI Ethics, Fairness & Society

Citations in the corpus

The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “…harms (Blodgett et al., 2020), not inclusive in framing (Dev et al., 2021b), ambiguous about what bias is measured (Blodgett et al., 2021; Goldfarb-Tarrant et al., 2023), not correlated in their findings of bias across intrinsic versus extrinsic techniques (GoldfarbTarrant et al., 2021;…”