Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation

By Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, Jason Aaron Edward Weston. 2019

236 citations recorded by the source corpus.

Models often easily learn biases present in the training data, and their predictions directly reflect this bias. We analyze gender bias in dialogue data, and examine how this bias is actually amplified in subsequent generative chit-chat dialogue models. We measure gender bias in six existing dialogue datasets, and focus on the most biased one, the multi-player text-based fantasy adventure dataset LIGHT, as a testbed for our bias mitigation techniques. The LIGHT dataset is highly imbalanced with respect to gender, containing predominantly male characters, likely because it is entirely collected by crowdworkers and reflects common biases that exist in fantasy or medieval settings. We consider three techniques to mitigate gender bias: counterfactual data augmentation, targeted data collection, and bias controlled training. We show that our proposed techniques mitigate gender bias in LIGHT by balancing the genderedness of generated dialogue utterances and are particularly effective in combination. We quantify performance using various evaluation methods---such as quantity of gendered words, a dialogue safety classifier, and human studies---all of which show that our models generate less gendered, but equally engaging chit-chat responses.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Training language models to follow instructions with human feedback (2022, background): “There are many ways to mitigate these harms, including by fine-tuning on a small, value-targeted dataset (Solaiman and Dennison, 2021), filtering the pretraining dataset (Ngo et al., 2021), or human-in-the-loop data collection (Dinan et al., 2019; Xu et al., 2020).”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “Vague/unstated Rudinger et al. (2018); Webster et al. (2018); Dinan et al. (2019); Florez (2019); Jumelet et al. (2019); Lauscher et al. (2019); Liang et al. (2019); Maudslay et al. (2019); May et al. (2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao…”
LaMDA: Language Models for Dialog Applications (2022, background): “Dialog models [84] can learn, and even amplify, biases in the training data.”
TruthfulQA: Measuring How Models Mimic Human Falsehoods (2021, background): “Increasing the safety and alignment of pre-trained models remains a challenging problem (Dinan et al., 2020; Tamkin et al., 2021; Xu et al., 2020; Solaiman and Dennison, 2021; McGuffie and Newhouse, 2020).”
Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “For autocomplete generation, Vig et al. (2020) analyze GPT-2 variants through a causal mediation analysis, ﬁnding that larger models contain more gender bias, and bias tends to be concentrated in a small number of neurons and attention heads.”
OPT: Open Pre-trained Transformer Language Models (2022, methodology): “We additionally evaluated OPT-175B on the ConvAI2 hidden test set, which has never been publicly released, and achieved 10.7 ppl and .185 UF1, matching the performance of the validation set.”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, methodology): “Augmentation with synthetically generated data has also been explored for gender bias mitigation in dialogue (Dinan et al., 2020; Liu et al., 2020) and translation”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Many strategies have been developed to address some of these harms (e.g., [58, 4, 48, 36, 34, 19, 60]).”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, methodology): “In particular, many works focus on generative models (Dinan et al., 2020a,b; Xu et al., 2021b; Kirk et al., 2021; Sheng et al., 2021b; Nozza et al., 2021; Renduchintala et al., 2021; Baheti et al., 2021; Perez et al., 2022), which are well known to pose unique challenges for automatic evaluation…”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “As in §4, we use the multi-dimensional gender bias classifier from Dinan et al. (2020b) to measure the amount of gender bias in conversation turns from Speaker A and B for 920,000 self-chats generated by each of our de-biased models.”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “This might be because the safety classiﬁer was trained to identify dialog utterances that are “ not OK to send in a friendly conversation with someone you just met online ”, which may encapsulate more than just toxic responses (Dinan et al., 2019b).”
Recipes for Safety in Open-domain Chatbots (2020, methodology): “…collection setup are given in Appendix A. Figure 1 demonstrates how this adversarial set-up differs from the “Build-it, Break-it, Fix-it” set-up from Dinan et al. (2019b): namely, in the former, the “breaker” (or adversarial user) tries to break a classiﬁer by submitting human-authored…”
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion (2022, background): “Further, generations can include toxic language and bias, especially with certain contexts and topics (Xu et al., 2020; Dinan et al., 2020).”
Towards Understanding and Mitigating Social Biases in Language Models (2021, background): “, 2020), data augmentation or collection (Dinan et al., 2020), and different objective functions (Qian et al.”
GeDi: Generative Discriminator Guided Sequence Generation (2020, background): “Other work has focused on removing gender bias from language models (Bordia and Bowman, 2019; Dinan et al., 2020; Bolukbasi et al., 2016).”
Does Gender Matter? Towards Fairness in Dialogue Systems (2019, background): “In the work (Dinan et al., 2019a), the authors examine gender bias in both dialogue datasets and generative dialogue models.”
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, methodology): “Finally, we add the Funpedia task (Dinan et al., 2020b) – which involves learning to produce an engaging dialogue utterance given a wikipedia sentence – and the LIGHT (Urbanek et al., 2019) and LIGHT WILD (Shuster et al., 2021b) tasks – which are open-domain dialogue tasks grounded in a medieval…”
Recipes for Building an Open-Domain Chatbot (2020, methodology): “For example, the ConvAI2 dataset (Zhang et al., 2018) focuses on personality and engaging the other speaker, Empa-thetic Dialogues (Rashkin et al., 2019) focuses on empathy, and Wizard of Wikipedia (Dinan et al., 2019c) focuses on knowledge.”