Does Gender Matter? Towards Fairness in Dialogue Systems

By Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, Jiliang Tang. 2019

156 citations recorded by the source corpus.

Recently there are increasing concerns about the fairness of Artificial Intelligence (AI) in real-world applications such as computer vision and recommendations. For example, recognition algorithms in computer vision are unfair to black people such as poorly detecting their faces and inappropriately identifying them as “gorillas”. As one crucial application of AI, dialogue systems have been extensively applied in our society. They are usually built with real human conversational data; thus they could inherit some fairness issues which are held in the real world. However, the fairness of dialogue systems has not been well investigated. In this paper, we perform a pioneering study about the fairness issues in dialogue systems. In particular, we construct a benchmark dataset and propose quantitative measures to understand fairness in dialogue models. Our studies demonstrate that popular dialogue models show significant prejudice towards different genders and races. Besides, to mitigate the bias in dialogue systems, we propose two simple but effective debiasing methods. Experiments show that our methods can reduce the bias in dialogue systems significantly. The dataset and the implementation are released to foster fairness research in dialogue systems.

Subfields: Artificial Intelligence, Language and Linguistics, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, methodology): “Finally, we actually categorized the motivation for Liu et al. (2019) (i.e., the last row in Table 3) as “Questionable correlations” due to a sentence elsewhere in the paper; had the paragraph we quoted been presented without more detail, we would have categorized the motivation as “Vague/unstated.””
LaMDA: Language Models for Dialog Applications (2022, methodology): “The method we use in this paper follows previous attempts to tackle these issues by training separate layers to detect unsafe output [17, 86, 18, 79].”
Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “based mitigation techniques rely on regularization (Bordia and Bowman, 2019; Huang et al., 2020; Liu et al., 2020a; Saunders and Byrne, 2020).”
OPT: Open Pre-trained Transformer Language Models (2022, methodology): “The pre-training corpus contains a concatenation of datasets used in RoBERTa (Liu et al., 2019b), the Pile (Gao et al., 2021a), and PushShift.io Red-dit (Baumgartner et al., 2020; Roller et al., 2021).”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, methodology): “Augmentation with synthetically generated data has also been explored for gender bias mitigation in dialogue (Dinan et al., 2020; Liu et al., 2020) and translation”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “…standard training datasets for dialogue models in the AI literature contain social biases or demographic imbalances, and it has been established that models learn them (Dixon et al., 2018; Bordia and Bowman, 2019; Lee et al., 2019; Dinan et al., 2020a,b; Liu et al., 2020a,b; Sheng et al., 2021).”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “These biases can result in toxicity being associated with certain words, such as profanities or identity terms (Dinan et al., 2019b; Dixon et al., 2018), or language varieties, such as African American English (AAE) (Liu et al., 2019; Sap et al., 2019).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design (2022, background): “However, these classifiers themselves can have issues with bias, e.g., by learning undesirable correlations that tie toxicity to identity terms (Dixon et al., 2018; Nozza et al., 2021, 2022), or language varieties, such as African American English (Liu et al., 2019; Sap et al., 2019).”
Towards Understanding and Mitigating Social Biases in Language Models (2021, background): “, 2018), image retrieval (Otterbacher, 2018), and dialog (Liu et al., 2020).”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, methodology): “A straightforward solution for gender bias in embeddings is Counterfactual Data Augmentation (CDA) (Hall Maudslay et al., 2019; Zmigrod et al., 2019; Liu et al., 2019).”
Red Teaming Language Models with Language Models (2022, background): “…find inputs that elicit inaccurate predictions from text classifiers (Hosseini et al., 2017; Ebrahimi et al., 2018; Behjati et al., 2019, inter alia) and offensive text from LMs (Wallace et al., 2019; He and Glass, 2019; Liu et al., 2019; Song et al., 2020; Liu et al., 2020; Yu and Sagae, 2021).”
Multi-Dimensional Gender Bias Classification (2020, background): “training corpora have been found to be amplified in machine learning models (Lee et al., 2019; Dinan et al., 2019a; Liu et al., 2019).”
Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models (2020, background): “However, with the wide application of neural dialogue models, the ethical challenges they bring are attracting more and more attention (Henderson et al., 2017; Liu et al., 2019a; Dinan et al., 2019).”
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020, methodology): “We use a predefined vocabulary of gender words released in the appendix of (Liu et al., 2019a).”
On Measures of Biases and Harms in NLP (2021, background): “For the similar task of dialogue generation, Liu et al. (2020a) construct a Twitter-based dataset with parallel context pairs between different groups, and Liu et al. (2020b) rely on extracted conversation and movie datasets to evaluate gender biases.”