The Variational Fair Autoencoder

By Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, Richard S. Zemel. 2015

682 citations recorded by the source corpus.

We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible. Our model is based on a variational autoencoding architecture with priors that encourage independence between sensitive and latent factors of variation. Any subsequent processing, such as classification, can then be performed on this purged latent representation. To remove any remaining dependencies we incorporate an additional penalty term based on the "Maximum Mean Discrepancy" (MMD) measure. We discuss how these architectures can be efficiently trained on data and show in experiments that this method is more effective than previous work in removing unwanted sources of variation while maintaining informative latent representations.

Subfields: Safety Research, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, methodology): “We aim towards counterfactual fairness by debiasing the latent representation of inputs in the language models, contributing to a family of methods to learn fair representations (Beutel et al., 2017; Zemel et al., 2013; Creager et al., 2019; Edwards and Storkey, 2016; Louizos et al., 2016) and enfor”
Does Object Recognition Work for Everyone? (2019, background): “For instance, statistical parity [5, 6, 14, 23–25, 30, 49] states that in binary-classification settings, members of different groups should have the same chance of receiving a positive prediction.”
Counterfactual Fairness in Text Classification through Robustness (2018, methodology): “d on debiasing methods to address these gaps. Many methods have been proposed to address group fairness issues, such as re-calibrating score functions [5], adversarially learning fair representations [1, 14, 22], data rebalancing [2], and data augmentation using swaps of gender terms [18]. Less sign”
Counterfactual Fairness (2017, background): “These include fairness through unawareness [12], individual fairness [10, 16, 24, 38], demographic parity/disparate impact [36], and equality of opportunity [14, 37].”
Neural Variational Inference for Text Processing (2015, background): “The recent variants of generative auto-encoder (Louizos et al., 2015; Makhzani et al., 2015) are also very competitive.”
Deep Variational Information Bottleneck (2017, background): “Finally, the variational fair autoencoder of Louizos et al. (2016) shares with our paper the idea of ignoring parts of the input.”
NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, background): “A similar nonadversarial approach that still uses a dependency measure of maximum mean discrepancy (MMD) for learning a variational autoencoder is in [37, 38].”
Learning Adversarially Fair and Transferable Representations (2018, methodology): “Louizos et al. (2016) give a method for learning fair representations with deep generative models by using maximum mean discrepancy (Gretton et al., 2007) to eliminate disparities between the two sensitive groups.”
Causal Effect Inference with Deep Latent-Variable Models (2017, neutral): “Since their recent introduction, VAEs have been shown to be remarkably successful in capturing latent structure across a wide-range of previously difficult problems, such as modeling images [19], volumes [24], time-series [10] and fairness [34].”
Wasserstein Fair Classification (2019, background): “…that do not contain sensitive information during training (Beutel et al., 2017; Calders et al., 2009; Calmon et al., 2017; Edwards and Storkey, 2016; Feldman et al., 2015; Fish et al., 2015; Kamiran and Calders, 2009, 2012; Louizos et al., 2016; Zemel et al., 2013; ˇZliobaite et al., 2011).”
Flexibly Fair Representation Learning by Disentanglement (2019, background): “One promising approach is fair representation learning: the design of neural networks using learning objectives that satisfy certain fairness or parity constraints in their outputs (Zemel et al., 2013; Louizos et al., 2016; Edwards & Storkey, 2016; Madras et al., 2018).”
Unsupervised Adversarial Invariance (2018, methodology): “Domain adaptation has been treated as an invariance induction task in recent literature [6, 14] where the goal is to make the prediction task invariant to the “domain” information.”
FairGAN: Fairness-aware Generative Adversarial Networks (2018, background): “ata other than the label, such as the Preferential Sampling [14] and the Disparate Impact Removal [9]. In addition, some methods prevent discrimination by learning fair representations from a dataset [8, 18, 23]. In this work, instead of removing the discrimination from the existing dataset, we focu”
Causal Reasoning for Algorithmic Fairness (2018, methodology): “Demographic parity has been used, for several purposes, in the following works: [14, 19, 20, 24, 42, 43].”
Invariant Representations without Adversarial Training (2018, methodology): “Maj. Class 0.725 0.695 VFAE [15] 0.717 0.720 Xie et al. [22] 0.811 0.695 Proposed 0.698 0.710 Adult Dataset Adv.”
Path-Specific Counterfactual Fairness (2018, methodology): “In order to ensure that ˜ q ( H | A ) does not depend on A , we follow a MMD penalization approach as in Louizos et al. (2016).”
On Fairness and Calibration (2017, background): “, under most frameworks of fairness, there is a trade-off between algorithmic performance and non-discrimination [4, 9, 19, 39]. Several works approach fairness through the lens of Statistical Parity [6, 7, 14, 20, 22, 23, 29, 38]. Under this deﬁnition, group membership should not affect the predict”
On the Fairness of Causal Algorithmic Recourse (2020, background): “While there are many different statistical notions of fairness [1–7], these are sometimes mutually incompatible [5] and it has been argued that discrimination, at its heart, corresponds to a causal influence of a protected attribute on the prediction, thus making fairness a fundamentally causal prob”