RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models

By Soumya Barikeri, Anne Lauscher, Ivan Vulic, Goran Glavas. 2021

225 citations recorded by the source corpus.

Text representation models are prone to exhibit a range of societal biases, reflecting the non-controlled and biased nature of the underlying pretraining data, which consequently leads to severe ethical issues and even bias amplification. Recent work has predominantly focused on measuring and mitigating bias in pretrained language models. Surprisingly, the landscape of bias measurements and mitigation resources and methods for conversational language models is still very scarce: it is limited to only a few types of bias, artificially constructed resources, and completely ignores the impact that debiasing methods may have on the final perfor mance in dialog tasks, e.g., conversational response generation. In this work, we present REDDITBIAS, the first conversational data set grounded in the actual human conversations from Reddit, allowing for bias measurement and mitigation across four important bias dimensions: gender,race,religion, and queerness. Further, we develop an evaluation framework which simultaneously 1)measures bias on the developed REDDITBIAS resource, and 2)evaluates model capability in dialog tasks after model debiasing. We use the evaluation framework to benchmark the widely used conversational DialoGPT model along with the adaptations of four debiasing methods. Our results indicate that DialoGPT is biased with respect to religious groups and that some debiasing techniques can remove this bias while preserving downstream task performance.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, methodology): “, 2020a); religion, race, gender, and orientation (Barikeri et al., 2021); or occupations (Kirk et al.”
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, background): “, 2020; Smith and Williams, 2021) or several axes of demographic axis more broadly (Barikeri et al., 2021; Perez et al., 2022; Smith et al., 2022).”
Using In-Context Learning to Improve Dialogue Safety (2023, background): “For example, these models often exhibit social biases (Barikeri et al., 2021; Dinan et al., 2020) and inappropriately align them-”
Bias and Fairness in Large Language Models: A Survey (2023, background): “RedditBias (Barikeri et al., 2021) introduces a conversational dataset generated from Reddit conversations to assess stereotypes between dominant and minoritized groups along the dimensions of gender, race, religion, and queerness.”
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2021, background): “, 2018), stereotype (Barikeri et al., 2021) and political prudence (Bang et al.”
Pipelines for Social Bias Testing of Large Language Models (2022, background): “The majority of work focused on gender bias, and only a few investigated an extensive range of targets (Nangia et al., 2020; Nadeem et al., 2021; Ousidhoum et al., 2021; Barikeri et al., 2021).”
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models (2021, background): “For instance, the CDA, Dropout, and INLP GPT-2 models obtain higher average GLUE scores than the baseline model.”
Measuring Harmful Sentence Completion in Language Models for LGBTQIA+ Individuals (2022, methodology): “To collect this set, we started from the queerness term set published by Barikeri et al. (2021).”
“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation (2023, background): “[4] Soumya Barikeri, Anne Lauscher, Ivan Vulić, and Goran Glavaš.”
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark (2022, background): “…toxicity and offensiveness (Baheti et al., 2021; Cercas Curry and Rieser, 2018; Dinan et al., 2021), bias (Henderson et al., 2018; Liu et al., 2020; Barikeri et al., 2021; Lee et al., 2019), privacy (Weidinger et al., 2021), sensitive topics (Xu et al., 2020; Sun et al., 2022), and moral…”
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models (2023, methodology): “To obtain target and attribute terms to cover more biases in texts, we collect target and attribute terms according to many previous NLP works on social biases (Nangia et al., 2020; Flekova et al., 2016; Barikeri et al., 2021), as well as sociology literature (Greenwald et al.”
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines (2022, background): “Dialogue Safety is an important concern for conversational models, as they can generate harmful content, exhibit social biases, and align themselves with offensive statements (Xu et al., 2021b; Ba-heti et al., 2021; Barikeri et al., 2021; Dinan et al., 2022).”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “In general, these pipelines consist of the following steps: language identification [124, 9], detoxification [87, 48, 171, 186], debiasing [179, 21, 16],”
Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models (2023, methodology): “Various methods have been developed to measure social bias and stereotypes in large language models (Dinan et al., 2020; Nangia et al., 2020; Nadeem et al., 2021; Schick et al., 2021; Barikeri et al., 2021; Kirk et al., 2021; Smith et al., 2022; An et al., 2023, inter alia).”
A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, unknown): “…topic[16,73,141,153],persona[15,76] DatatoText Input: [78],genderbias[28,104],subjectivebias[101], socialbias[4,121],sentimentbias[51],toxicity[62,74]) FormatControl Input:DesiredFormat,prompttext Output:Textinpre-definedformation Format[66,70,120,122] We…”
"I'm sorry to hear that": finding bias in language models with a holistic descriptor dataset (2022, methodology): “…frequency of different words on a word list, for example, those signifying gender (Dinan et al., 2020a); religion, race, gender, and orientation (Barikeri et al., 2021); or occupations (Kirk et al., 2021b In the following sections, we motivate using H OLISTIC B IAS in measurements of token…”