Towards Debiasing Sentence Representations

By Percy Liang, Irene Z Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, Louis-philippe Morency. 2020

291 citations recorded by the source corpus.

As natural language processing methods are increasingly deployed in real-world scenarios such as healthcare, legal systems, and social science, it becomes necessary to recognize the role they potentially play in shaping social biases and stereotypes. Previous work has revealed the presence of social biases in widely used word embeddings involving gender, race, religion, and other social constructs. While some methods were proposed to debias these word-level embeddings, there is a need to perform debiasing at the sentence-level given the recent shift towards new contextualized sentence representations such as ELMo and BERT. In this paper, we investigate the presence of social biases in sentence-level representations and propose a new method, Sent-Debias, to reduce these biases. We show that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks such as sentiment analysis, linguistic acceptability, and natural language understanding. We hope that our work will inspire future research on characterizing and removing social biases from widely adopted sentence representations for fairer NLP.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “Vague/unstated Rudinger et al. (2018); Webster et al. (2018); Dinan et al. (2019); Florez (2019); Jumelet et al. (2019); Lauscher et al. (2019); Liang et al. (2019); Maudslay et al. (2019); May et al. (2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao…”
LaMDA: Language Models for Dialog Applications (2022, background): “, [74, 75, 76, 77, 78, 79, 80, 81, 82]), meaningfully addressing these issues remains an active research area.”
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020, methodology): “Liang et al. (2020) introduce a debiasing algorithm and they report lower bias scores on the SEAT while maintaining downstream task performance on the GLUE benchmark (Wang et al., 2018).”
Towards Understanding and Mitigating Social Biases in Language Models (2021, methodology): “Recent work has also extended this analysis towards measuring (Tan and Celis, 2019; Guo and Caliskan, 2020; Kurita et al., 2019) and mitigating (Liang et al., 2020; Ravfogel et al., 2020) bias in contextual embeddings such as ELMo (Peters et al., 2018), BERT (Devlin et al., 2019), and GPT (Radford…”
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets (2021, background): “Previously proposed debiasing methods include [6]’s foundational work to debias word embeddings; [29]’s use of product of experts to train a model to avoid dataset biases; [39]’s human-and-model-in-the-loop technique to better train and evaluate models without toxic and unwanted behavior; [23]’s use”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, methodology): “Despite the relatively ample literature on gender debiasing for word-level representations, very little work has focused on sentence representations (Liang et al., 2019; Liu et al., 2019; Sheng et al., 2019; Lee et al., 2019b).”
RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021, methodology): “For (i), there has been some work on debiasing PLMs (Liang et al., 2020),”
A Survey on Bias and Fairness in Natural Language Processing (2022, neutral): “Bias is measured in such cases using the SEAT test (Liang et al., 2019; May et al., 2019).”
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021, methodology): “…been shown to encode and amplify a range of stereo-typical human biases (e.g., gender or racial biases) (Zhao et al., 2019; Basta et al., 2019; Liang et al., 2020a,b), much like their static embedding pre-decessors (Bolukbasi et al., 2016; Caliskan et al., 2017; Dev and Phillips, 2019; Gonen…”
StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer (2021, background): “…focused on building dialog systems (Celikyilmaz et al., 2018), virtual assistants (Cooper et al., 2004), and intelligent agents (Kim et al., 2013; Liang et al., 2020a; Pittermann et al., 2010) that can communicate their intentions with specific styles for different situations, target audiences,…”
Efficient Gender Debiasing of Pre-trained Indic Language Models (2022, methodology): “Liang et al. (2020) used an approach similar to hard-debiasing for context-based representations.”
Handling Bias in Toxic Speech Detection: A Survey (2022, background): “[48] extended this idea to contextual sentence embeddings.”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “Prior work has described how quality-of-service harms are especially likely when system inputs rely on biometric data (e.g., facial features, skin tone, or voice), such as computer vision [46, 176], natural language processing [113, 131, 173], and speech recognition systems [124, 143].”
Theories of “Gender” in NLP Bias Research (2022, background): “[85] Paul Pu Liang, Irene Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency.”
Text and Code Embeddings by Contrastive Pre-Training (2022, methodology): “…training data and pre-trained models, and (b) developing and improving methods for mitigating encoded bias, including ﬁne-tuning to reduce bias in pre-trained models (Caliskan et al., 2017; May et al., 2019; Bolukbasi et al., 2016; Liang et al., 2020; Park et al., 2018; Solaiman & Dennison, 2021).”
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021, methodology): “Though previous work has addressed biases in frozen PTLM or word embeddings (Bolukbasi et al., 2016; Zhou et al., 2019; Bhardwaj et al., 2020; Liang et al., 2020; Ravfogel et al., 2020), for example by measuring associations between gender and occupations in an embedding space, they do not study…”
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models (2021, methodology): “Because of these ﬁndings, an increasing amount of research has focused on developing techniques to mitigate these biases (Liang et al., 2020; Ravfogel et al., 2020; Webster et al., 2020; Kaneko and Bollegala, 2021; Schick et al., 2021; Lauscher et al., 2021).”