Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations

By Alex Beutel, Jilin Chen, Zhe Zhao, Ed H. Chi. 2017

473 citations recorded by the source corpus.

How can we learn a classifier that is "fair" for a protected or sensitive group, when we do not know if the input to the classifier belongs to the protected group? How can we train such a classifier when data on the protected group is difficult to attain? In many settings, finding out the sensitive input attribute can be prohibitively expensive even during model training, and sometimes impossible during model serving. For example, in recommender systems, if we want to predict if a user will click on a given recommendation, we often do not know many attributes of the user, e.g., race or age, and many attributes of the content are hard to determine, e.g., the language or topic. Thus, it is not feasible to use a different classifier calibrated based on knowledge of the sensitive attribute. Here, we use an adversarial training procedure to remove information about the sensitive attribute from the latent representation learned by a neural network. In particular, we study how the choice of data for the adversarial training effects the resulting fairness properties. We find two interesting results: a small amount of data is needed to train these adversarial models, and the data distribution empirically drives the adversary's notion of fairness.

Subfields: Artificial Intelligence, Safety Research, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, background): “…fairness by debiasing the latent representation of inputs in the language models, contributing to a family of methods to learn fair representations (Beutel et al., 2017; Zemel et al., 2013; Creager et al., 2019; Edwards and Storkey, 2016; Louizos et al., 2016) and enforcing independence between…”
Gender Bias in Coreference Resolution (2018, background): “Further work on debias-ing models with adversarial learning is explored by Beutel et al. (2017) and Zhang et al. (2018).”
Counterfactual Fairness in Text Classification through Robustness (2018, methodology): “Many methods have been proposed to address group fairness issues, such as re-calibrating score functions [6], adversarially learning fair representations [1, 14, 23], data rebalancing [3], and data augmentation using swaps of gender terms [18].”
Mitigating Gender Bias in Natural Language Processing: Literature Review (2019, background): “…to NLP; other ﬁelds in computer science such as data mining, machine learning, and security also study gender bias (Calders and Verwer, 2010; Feldman et al., 2015; Hardt et al., 2016; Misra et al., 2016; Kleinberg et al., 2016; Pleiss et al., 2017; Beutel et al., 2017; Kilbertus et al., 2017).”
Mitigating Unwanted Biases with Adversarial Learning (2018, methodology): “[2] apply an adversarial training method to achieve eqality of opportunity in cases when the output variable is discrete.”
Learning Adversarially Fair and Transferable Representations (2018, neutral): “Beutel et al. (2017) explored the particular fairness levels achieved by the algorithm from Edwards & Storkey (2016), and demonstrated that they can vary as a function of the demographic unbalance of the training data.”
Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction (2018, methodology): “Beutel et al. (2017) used an adversary on a shared hidden layer to satisfy parity for salary prediction, and Ganin et al. (2016) used a domain classifier and reverse gradient on a hidden layer to remove domain correlation. Recently, Zhang et al. (2018) used a predictor and adversary with an addition”
Wasserstein Fair Classification (2019, methodology): “…consists in pre-processing the data to remove bias, or in extracting representations that do not contain sensitive information during training (Beutel et al., 2017; Calders et al., 2009; Calmon et al., 2017; Edwards and Storkey, 2016; Feldman et al., 2015; Fish et al., 2015; Kamiran and…”
The Price of Fair PCA: One Extra Dimension (2018, background): “The ﬁrst piece is a broad line of work [Zemel et al., 2013; Beutel et al., 2017; Calmon et al., 2017; Madras et al., 2018; Zhang et al., 2018] that aims to design representations which will be conditionally independent of the protected attribute, while retaining as much information as possible (and…”
The Frontiers of Fairness in Machine Learning (2018, background): “There is also a growing literature on using adversarial learning to achieve group fairness in the form of statistical parity or false positive/false negative rate balance [ES15, BCZC17, ZLM18, MCPZ18].”
Taking Advantage of Multitask Learning for Fair Classification (2018, neutral): “This is in contrast to other approaches which impose the fairness constraint as a post-processing step [10 19 20 31] or by modifying the data representation before employing standard machine learning methods [1 12 22–24 39].”
FairGAN: Fairness-aware Generative Adversarial Networks (2018, background): “However, many studies have shown that machine learning models have biased performance against the protected group [2], [3].”
Adversarial Removal of Demographic Attributes from Text Data (2018, methodology): “Many other deﬁnitions of fairness exist, including demographic parity , equality of odds and equality of opportunity (see e.g. discussion in (Hardt et al., 2016; Beutel et al., 2017)).”
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification (2019, methodology): “In addition, several works, such as [10], [1], [6] and [5] provide techniques intending to mitigate bias in machine learning models.”
Measuring and Reducing Gendered Correlations in Pre-trained Models (2020, positive): “Another popular technique uses adversarial losses in order to remove demographic information from learned representations (Louizos et al., 2015; Edwards and Storkey, 2015; Beutel et al., 2017; Zhang et al., 2018; Elazar and Goldberg, 2018; Madras et al., 2018).”
Empirical Risk Minimization under Fairness Constraints (2018, background): “e divided in three families. Methods in the ﬁrst family modify a pretrained classiﬁer in order to increase its fairness properties while maintaining as much as possible the classiﬁcation performance: [15, 16, 2, 17] are examples of these methods but no consistency property nor comparison with state-”
Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach (2019, background): “More recent approaches [3, 20, 1, 18] have used explicit adversarial networks to measure information content of sensitive attributes.”