Intersectional Bias in Causal Language Models

By Liam Magee, Lida Ghahremanlou, Karen Soldatic, S. Robertson. 2021

45 citations recorded by the source corpus.

To examine whether intersectional bias can be observed in language generation, we examine \emph{GPT-2} and \emph{GPT-NEO} models, ranging in size from 124 million to ~2.7 billion parameters. We conduct an experiment combining up to three social categories - gender, religion and disability - into unconditional or zero-shot prompts used to generate sentences that are then analysed for sentiment. Our results confirm earlier tests conducted with auto-regressive causal models, including the \emph{GPT} family of models. We also illustrate why bias may be resistant to techniques that target single categories (e.g. gender, religion and race), as it can also manifest, in often subtle ways, in texts prompted by concatenated social categories. To address these difficulties, we suggest technical and community-based approaches need to combine to acknowledge and address complex and intersectional language model bias.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

BBQ: A hand-built bias benchmark for question answering (2021, background): “Measuring Bias in NLP Several studies have investigated the prevalence of biases in NLP models (Caliskan et al., 2017; May et al., 2019; Bordia and Bowman, 2019; Davidson et al., 2019; Magee et al., 2021), with many focusing on cases of models exhibiting stereotyping behavior.”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “…works have highlighted the importance of studying intersec-723 tional bias (Bender et al., 2021; Buolamwini and Gebru, 2018; Field et al., 2021; Guo et al., 2019; Crenshaw, 1991) but little research has been conducted around addressing such biases (Magee et al., 2021; Guo and Caliskan, 2021).”
Datasheet for the Pile (2022, methodology): “Pile-CC: Papers that train models on datasets that include the Pile-CC subset of the Pile include Luo et al. [2021], Kharya and Alvi [2021], Askell et al. [2021]. While this data has likely been used by other researchers for various purposes, we are unaware of any uses that would be directly compara”