Learning unbiased features

By Yujia Li, Kevin Swersky, Richard S. Zemel. 2014

73 citations recorded by the source corpus.

A key element in transfer learning is representation learning; if representations can be developed that expose the relevant factors underlying the data, then new tasks and domains can be learned readily based on mappings of these salient factors. We propose that an important aim for these representations are to be unbiased. Different forms of representation learning can be derived from alternative definitions of unwanted bias, e.g., bias to particular tasks, domains, or irrelevant underlying data dimensions. One very useful approach to estimating the amount of bias in a representation comes from maximum mean discrepancy (MMD) [5], a measure of distance between probability distributions. We are not the first to suggest that MMD can be a useful criterion in developing representations that apply across multiple domains or tasks [1]. However, in this paper we describe a number of novel applications of this criterion that we have devised, all based on the idea of developing unbiased representations. These formulations include: a standard domain adaptation framework; a method of learning invariant representations; an approach based on noise-insensitive autoencoders; and a novel form of generative model.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Adversarial Invariance (2018, methodology): “This research area has, therefore, garnered tremendous interest recently [6, 13, 14, 19].”
The Variational Fair Autoencoder (2015, methodology): “Following Li et al. (2014), we used the same training, test set and no validation set.”
Controllable Invariance through Adversarial Feature Learning (2017, background): “…the bilingual encoder-decoder even though multi-lingual Method Accuracy of classifying factor s Accuracy of classifying label y Logistic regression 0.96 0.78 NN + MMD [Li et al., 2014] - 0.82 VFAE [Louizos et al., 2016] 0.57 0.85 Ours 0.57 0.89 Table 2: Results on Extended Yale B dataset.”
Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach (2019, methodology): “The MaxEnt-ARL achieves an accuracy of 89% for identity classification (same as ML-ARL) while outperforming MMD (82”
Invariant Representations through Adversarial Forgetting (2019, methodology): “The Adult (Dheeru and Karra Taniskidou 2017) and German (Dheeru and Karra Taniskidou 2017) datasets have been popularly employed in fairness settings (Li, Swersky, and Zemel 2014; Louizos et al. 2016; Moyer et al. 2018; Xie et al. 2017).”