Counterfactual Fairness in Text Classification through Robustness

By Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, Alex Beutel. 2018

294 citations recorded by the source corpus.

In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different? Toxicity classifiers demonstrate a counterfactual fairness issue by predicting that "Some people are gay" is toxic while "Some people are straight" is nontoxic. We offer a metric, counterfactual token fairness (CTF), for measuring this particular form of fairness in text classifiers, and describe its relationship with group fairness. Further, we offer three approaches, blindness, counterfactual augmentation, and counterfactual logit pairing (CLP), for optimizing counterfactual token fairness during training, bridging the robustness and fairness literature. Empirically, we find that blindness and CLP address counterfactual token fairness. The methods do not harm classifier performance, and have varying tradeoffs with group fairness. These approaches, both for measurement and optimization, provide a new path forward for addressing fairness concerns in text classification.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Safety Research, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Safety Research

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…Cho et al. (2019); Davidson et al. (2019); Dev et al. (2019); Dinan et al. (2019); Fisher (2019); Florez (2019); Font and Costa-jussà (2019); Garg et al. (2019); Huang et al. (2019); Liu et al. (2019); Nozza et al. (2019); Prabhakaran et al. (2019); Qian et al. (2019); Sap et al. (2019);…”
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, background): “counterfactual fairness works above by considering counterfactual fairness (Garg et al., 2019) based on a simple causal graph representing the language model instead of the data-generating process.”
AI and the Everything in the Whole Wide World Benchmark (2021, methodology): “• System output analysis / behavioural testing (see Appendix A.2), including error analysis [e.g. van Miltenburg et al., 2021], disaggregated analysis [e.g. Raji and Buolamwini, 2019], and counterfactual analysis [e.g. Garg et al., 2019, Hutchinson et al., 2020].”
Carbon Emissions and Large Neural Network Training (2021, background): “Creating such models requires careful attention to issues of fairness and bias [Ben21, Gar19, Joh20, Kuc18, Mer19], but they also have the potential to benefit people everywhere.”
Re-imagining Algorithmic Fairness in India and Beyond (2021, background): “While other axes of discrimination and injustices such as disability status [91], age [59], and sexual orientation [76] have gotten some attention, biases relevant to other geographies and cultures are not explored (e.”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “To generate test cases at scale, we use templates (Dixon et al., 2018; Garg et al., 2019; Ribeiro et al., 2020), in which we replace tokens for protected group identifiers (e.”
Social Biases in NLP Models as Barriers for Persons with Disabilities (2020, methodology): “Following (Garg et al., 2019; Prabhakaran et al., 2019), we use the notion of perturbation, whereby the phrases for referring to people with disabilities, described above, are all inserted into the same slots in sentence templates.”
Re-contextualizing Fairness in NLP: The Case of India (2022, background): “…2016; Sun et al., 2019; Zhao et al., 2017) and racial biases (Sap et al., 2019; Davidson et al., 2019; Manzini et al., 2019), other axes of disparities such as ability (Hutchinson et al., 2020), age (Diaz et al., 2018), and sexual orientation (Garg et al., 2019) have also gotten some attention.”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “Some of these efforts augment humans (through guidelines, templates, programmatic generation of attacks, and various combinations thereof) to devise test cases that cause systems to fail [45, 46, 29, 21, 30, 55, 6, 23].”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “, to assess counterfactual notions of fairness (Garg et al., 2019; Kusner et al., 2017).”
Red Teaming Language Models with Language Models (2022, methodology): “Other efforts manually compose templates and code to generate test cases for specific failures (Jia and Liang, 2017; Dixon et al., 2018; Garg et al., 2019; Jiang and Bansal, 2019; Ribeiro et al., 2020).”
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021, background): “5We collect asymmetric counterfactuals (Garg et al., 2019) by sampling more Duplicate and Entailment examples in QQP and NLI to perturb, due to the difficulty of flipping other labels.”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, background): “cases, false negatives can be provoked by changing the ‘sensitive’ attribute of hateful content, such as changing the target from ‘gay’ to ‘black’ people (Garg et al., 2019).”
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification (2019, background): “th a variety of thresholds, the Equality Gap falls short for many purposes, as it evaluates the model only at one specific threshold. 1https://git.io/fhpcC The related work on counterfactual fairness [9] proposes a number of techniques to reduce classifiers affinity for assigning attributed features”
Towards Debiasing Sentence Representations (2020, background): “In particular, Zhao et al. (2019), Park et al. (2018), and Garg et al. (2019) are not able to perform post-hoc debiasing and require changing the data or underlying word embeddings and retraining which is costly.”
Perturbation Sensitivity Analysis to Detect Unintended Model Biases (2019, background): “Hence, fairness research in NLP has mostly focused on mentions of social identities (Dixon et al., 2018; Borkan et al., 2019; Garg et al., 2019), or on how social stereotypes impact semantic interpretation (Webster et al., 2018), and often rely heavily on annotated corpora.”
Measuring and Reducing Gendered Correlations in Pre-trained Models (2020, neutral): “to the training data directly (Zhao et al., 2018b; Garg et al., 2019).”
Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text (2020, methodology): “…increase in the work on fairness testing (Ma, Wang, and Liu; Holstein et al. 2019) which are capable of generating a large number of test-cases that capture the model’s ability to misclassify or remove unwanted bias around speciﬁc protected attributes (Huang et al. 2019), (Garg et al. 2019).”