Learning Controllable Fair Representations

By Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, Stefano Ermon. 2018

188 citations recorded by the source corpus.

Learning data representations that are transferable and are fair with respect to certain protected attributes is crucial to reducing unfair decisions while preserving the utility of the data. We propose an information-theoretically motivated objective for learning maximally expressive representations subject to fairness constraints. We demonstrate that a range of existing approaches optimize approximations to the Lagrangian dual of our objective. In contrast to these existing approaches, our objective allows the user to control the fairness of the representations by specifying limits on unfairness. Exploiting duality, we introduce a method that optimizes the model parameters as well as the expressiveness-fairness trade-off. Empirical evidence suggests that our proposed method can balance the trade-off between multiple notions of fairness and achieves higher expressiveness at a lower computational cost.

Subfields: Artificial Intelligence, Safety Research, Representation & Self-Supervised Learning, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Representation & Self-Supervised Learning

In AI Ethics, Fairness & Society

Citations in the corpus

Attributing Fair Decisions with Attention Interventions (2021, methodology): “Existing methods for fair machine learning include applying complex transformations to the data so that resulting representations are fair (Gupta et al. 2021; Moyer et al. 2018; Roy and Boddeti 2019; Jaiswal et al. 2020; Song et al. 2019), adding regularizers to incorporate fairness (Zafar et al.”
Flexibly Fair Representation Learning by Disentanglement (2019, background): “Several recent VAE variants have incorporated label information into latent vari-Flexibly able learning for image synthesis (Klys et al., 2018) and single-attribute fair representation learning (Song et al., 2019; Botros & Tomczak, 2018; Moyer et al., 2018).”
Invariant Representations without Adversarial Training (2018, neutral): “Contemporaneous to this work, another paper by Song et al. [20] proposes a similar information theory based bound.”
On the Fairness of Disentangled Representations (2019, background): “The authors of [84] instead propose an information-theoretic approach in which the mutual information between the data and the representation is maximized, while the one between the sensitive attributes and the representation is minimized.”
Guided Variational Autoencoder for Disentanglement Learning (2020, background): “Recent efforts in fairness disentanglement learning [9, 47] also bear some similarity, but there is still a large difference in formulation.”
Rényi Fair Inference (2019, methodology): “2019) used an adversarial approach that estimates the joint probability density function using a parameterized neural network.”
Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations (2021, methodology): “[38] Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon.”
Fair Generative Modeling via Weak Supervision (2019, background): “The vast majority of related work in this area is centered around fair and/or privacy preserving representation learning, which exploit tools from adversarial learning and information theory among others (Zemel et al., 2013; Edwards & Storkey, 2015; Louizos et al., 2015; Beutel et al., 2017; Song et”
You Don’t Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers’ Private Personas (2022, unknown): “Instead, we use pΨ(s|f(u)) to approximate q(s|f(u)) via minimizing their KL divergence and then we can obtain the following lower bound (Song et al., 2019): Eq(f(u)) DKL(q(s|f(u))||p(s)) ≥ Eq(f(u))[log pΨ(s|f(u))− log p(s)].”
Fairness in Machine Learning (2020, methodology): “Pre-Processing methods consist in transforming the training data to remove undesired biases [16, 21, 24, 28, 47, 51, 52, 53, 61, 64, 65, 66, 67, 86, 90, 91, 92, 112, 114, 116, 118, 121, 122, 127, 126, 164, 174, 188, 190, 191, 193].”
Overlearning Reveals Sensitive Attributes (2019, methodology): “For fairness, it is sufficient to censor the model’s final layer to ensure that the output is independent of the sensitive attributes or satisfies a specific fairness constraint [25, 26, 33, 39].”
Adhering, Steering, and Queering: Treatment of Gender in Natural Language Generation (2020, background): “In order to achieve fair representation, Song et al. [65] introduce two criteria: • expressiveness : maintain as much non-sensitive information as possible (see [44]).”
A Theory of Usable Information Under Computational Constraints (2020, methodology): “Existing methods obtain fair representations by optimizing against an “adversarial” discriminator so that the discriminator cannot predict U from Z (Edwards & Storkey, 2015; Louizos et al., 2015; Madras et al., 2018; Song et al., 2018).”
Multi-label Contrastive Predictive Coding (2020, methodology): “Nevertheless, our method is not agnostic to existing biases in the dataset, so there is a potential danger that any bias that are inherent in the data collection process are also exhibited in the learned representations, such as bias against minority groups [45].”