Mitigating Unwanted Biases with Adversarial Learning

By Brian Hu Zhang, Blake Lemoine, Margaret Mitchell. 2018

1,702 citations recorded by the source corpus.

Machine learning is a tool for building models that accurately represent input training data. When undesired biases concerning demographic groups are in the training data, well-trained models will reflect those biases. We present a framework for mitigating such biases by including a variable for the group of interest and simultaneously learning a predictor and an adversary. The input to the network X, here text or census data, produces a prediction Y, such as an analogy completion or income bracket, while the adversary tries to model a protected variable Z, here gender or zip code. The objective is to maximize the predictor's ability to predict Y while minimizing the adversary's ability to predict Z. Applied to analogy completion, this method results in accurate predictions that exhibit less evidence of stereotyping Z. When applied to a classification task using the UCI Adult (Census) Dataset, it results in a predictive model that does not lose much accuracy while achieving very close to equality of odds (Hardt, et al., 2016). The method is flexible and applicable to multiple definitions of fairness as well as a wide range of gradient-based learning models, including both regression and classification tasks.

Subfields: Artificial Intelligence, Demography, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Demography

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, background): “beddings (Zhang et al., 2018) as well as in image recognition models (Zhao et al.”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…et al. (2017); Burns et al. (2018); Dixon et al. (2018); Kiritchenko and Mohammad (2018); Lu et al. (2018); Park et al. (2018); Shen et al. (2018); Zhang et al. (2018); Badjatiya et al. (2019); Bhargava and Forsyth (2019); Cao and Daumé (2019); Cho et al. (2019); Davidson et al. (2019); Dev et…”
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (2019, background): “In addition, they encourage the representation of neutral-gender words (excluding the last coordinate) to be orthogonal to the gender direc- 3Another work in this spirit is that of Zhang et al. (2018), which uses an adversarial network to debias word embeddings.”
Gender Bias in Coreference Resolution (2018, background): “Further work on debias-ing models with adversarial learning is explored by Beutel et al. (2017) and Zhang et al. (2018).”
Re-imagining Algorithmic Fairness in India and Beyond (2021, neutral): “, [42, 218]), alongside testing strategies, e.”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, background): “Researchers in AI and machine learning have increasingly paid attention to algorithmic fairness and dataset and model biases [71, 11, 76, 72].”
Women also Snowboard: Overcoming Bias in Captioning Models (2018, background): “, age, gender, sexual orientation) in a fair manner is increasingly important as AI continues to become more ubiquitous [14,15,16,17].”
Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy (2019, background): “The downside of this solution is that balancing the dataset instead of releasing the image-level attribute annotations makes it impossible to evaluate the error rates of machine learning algorithms on demographic subsets of the data, as is common in the literature [10, 36, 82, 83].”
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems (2018, background): “Approaches on how to mitigate inappropriate biases (Schmidt, 2015; Bolukbasi et al., 2016; Kilbertus et al., 2017; Ryu et al., 2017; Speer, 2017; Zhang et al., 2018; Zhao et al., 2018) are also beyond the scope of this paper.”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, background): “been empirically verified in limited applications (Zhang et al., 2018; Zhao et al., 2017), and it is not clear that these methods can generalize to other tasks or models.”
Reducing Gender Bias in Abusive Language Detection (2018, methodology): “Some previous works (Beutel et al.; Zhang et al., 2018) employ adversarial training methods to make the classiﬁers unbiased toward certain variables.”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “An example is Adversarial Debiasing, [397] .”
A Survey on Bias and Fairness in Machine Learning (2019, methodology): “he real data denoted by P data(x,y,s)from the generated fake data denoted by P G(x,y,s). Fig. 5. Structure of FairGAN as proposed in [129]. Fig. 6. The architecture of adversarial network proposed in [76]. 20 Mehrabi et al. In addition to that, for achieving fairness constraints, such as statistical”
Learning Adversarially Fair and Transferable Representations (2018, methodology): “As a third baseline, we trained an adversarial model similar to the one proposed in (Zhang et al., 2018), where the adversary has access only to the classifier output Ŷ = g(Z) and the ground truth label (Transfer-Y-Adv), to investigate the utility of our adversary having access to the underlying rep”
Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction (2018, methodology): “Recently, Zhang et al. (2018) used a predictor and adversary with an additional projection term to satisfy parity or equality of odds with word embeddings and predicted salary.”
A Survey on Bias and Fairness in Natural Language Processing (2022, neutral): “Zhang et al. (2018) model a variation of GANs for mitigating biases.”
Wasserstein Fair Classification (2019, methodology): “Adversarial methods maximize the system ability to predict Y while minimizing the ability to predictA (Zhang et al., 2018).”