Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning

By Haochen Liu, Wentao Wang, Yiqi Wang, Hui Liu, Zitao Liu, Jiliang Tang. 2020

79 citations recorded by the source corpus.

Dialogue systems play an increasingly important role in various aspects of our daily life. It is evident from recent research that dialogue systems trained on human conversation data are biased. In particular, they can produce responses that reflect people's gender prejudice. Many debiasing methods have been developed for various NLP tasks, such as word embedding. However, they are not directly applicable to dialogue systems because they are likely to force dialogue models to generate similar responses for different genders. This greatly degrades the diversity of the generated responses and immensely hurts the performance of the dialogue models. In this paper, we propose a novel adversarial learning framework Debiased-Chat to train dialogue models free from gender bias while keeping their performance. Extensive experiments on two real-world conversation datasets show that our framework significantly reduces gender bias in dialogue models while maintaining the response quality. The implementation of the proposed framework is released.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Societal Biases in Language Generation: Progress and Challenges (2021, methodology): “CDA has been applied to datasets used for continued or fresh training in dialogue generation (Dinan et al., 2020a; Liu et al., 2020a) as well as machine translation (Saunders and Byrne, 2020; Costa-jussà and de Jorge, 2020; Stafanovičs et al., 2020).”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “…standard training datasets for dialogue models in the AI literature contain social biases or demographic imbalances, and it has been established that models learn them (Dixon et al., 2018; Bordia and Bowman, 2019; Lee et al., 2019; Dinan et al., 2020a,b; Liu et al., 2020a,b; Sheng et al., 2021).”
Does Gender Matter? Towards Fairness in Dialogue Systems (2019, methodology): “Moreover, based on the bias measurements in this work, there is a recent work (Liu et al., 2020a) introducing an adversarial learning framework Debiased-Chat to mitigate gender bias in neural dialogue models.”
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021, methodology): “…is to maintain an accurate account of the dialog belief state (i.e., information slots and their values provided by the user) at each turn of the conversation, combining the information from the current user utterance and the conversation history (Henderson et al., 2014; Mrkˇsi´c et al., 2017).”
Five sources of bias in natural language processing (2021, background): “For example debiasing embeddings for reducing gender bias in text classification (Prost et al., 2019), dialogue generation (Dinan et al., 2020; Liu et al., 2020), and machine translation (Font & Costa-jussà, 2019).”
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models (2023, methodology): “Moreover, existing works (Barikeri et al., 2021; Dinan et al., 2020; Liu et al., 2020b) only focus on English dialogue models.”
Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decoding (2021, unknown): “Debiasing Language Generation There are three main streams to debias NLG tasks: coun-terfactual data augmentation (Lu et al., 2020; Chen et al., 2018); training-time methods (Huang et al., 2020; Liu et al., 2020a,b; Kaneko and Bollegala, 2021; Pryzant et al., 2020); and inference-time methods.”
“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation (2023, background): “TGNB Harm Evaluations in LLMs Gender bias evaluation methods include toxicity measurements and word co-occurrence in OLG [23, 25, 37, 40, 59, 61].”
What do Bias Measures Measure? (2021, background): “Liu et al. (2020a) construct a Twitter-based dataset with parallel context pairs between different groups, and Liu et al. (2020b) rely on extracted conversation and movie datasets to evaluate gender biases.”
On Measures of Biases and Harms in NLP (2021, background): “For the similar task of dialogue generation, Liu et al. (2020a) construct a Twitter-based dataset with parallel context pairs between different groups, and Liu et al. (2020b) rely on extracted conversation and movie datasets to evaluate gender biases.”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Likewise, a commonsense knowledge graph was used by Liu et al. (2020c) to lead the conversation topic and make recommendations.”