Your fairness may vary: Pretrained language model fairness in toxic text classification

By Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Mikhail Yurochkin, Moninder Singh. 2021

61 citations recorded by the source corpus.

The popularity of pretrained language models in natural language processing systems calls for a careful evaluation of such models in down-stream tasks, which have a higher potential for societal impact. The evaluation of such systems usually focuses on accuracy measures. Our findings in this paper call for attention to be paid to fairness measures as well. Through the analysis of more than a dozen pretrained language models of varying sizes on two toxic text classification tasks (English), we demonstrate that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics. Specifically, we observe that fairness can vary even more than accuracy with increasing training data size and different random initializations. At the same time, we find that little of the fairness variation is explained by model size, despite claims in the literature. To improve model fairness without retraining, we show that two post-processing methods developed for structured, tabular data can be successfully applied to a range of pretrained language models. Warning: This paper contains samples of offensive text.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “5 As the model has skewed cumulative distribution towards uncertainty for the minority, any fairness metric defined on the output of such a model will also reflect this instability, and thus manifests as fairness variance in existing literature [3, 46, 49].”
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “…Nadeem et al., 2021) and expressed in specific tasks (Nangia et al., 2020; Kirk et al., 2021; Schramowski et al., 2022; Prabhumoye et al., 2021; Srinivasan and Bisk, 2021; Kirk et al., 2021; Parrish et al., 2021; Baldini et al., 2022; Czarnowska et al., 2021; Dev et al., 2021a; Zhao et al., 2021).”
An Empirical Study of the Non-Determinism of ChatGPT in Code Generation (2023, unknown): “This non-determinism (i.e., the inconsistency in the code candidates generated in different requests with identical prompts) 1 is an essential consideration when using LLM in practice [5].”