Learning to Pivot with Adversarial Networks

By Gilles Louppe, Michael Kagan, Kyle Cranmer. 2016

238 citations recorded by the source corpus.

Several techniques for domain adaptation have been proposed to account for differences in the distribution of the data used for training and testing. The majority of this work focuses on a binary domain label. Similar problems occur in a scientific context where there may be a continuous family of plausible data generation processes associated to the presence of systematic uncertainties. Robust inference is possible if it is based on a pivot -- a quantity whose distribution does not depend on the unknown values of the nuisance parameters that parametrize this family of data generation processes. In this work, we introduce and derive theoretical results for a training procedure based on adversarial networks for enforcing the pivotal property (or, equivalently, fairness with respect to continuous attributes) on a predictive model. The method includes a hyperparameter to control the trade-off between accuracy and robustness. We demonstrate the effectiveness of this approach with a toy example and examples from particle physics.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, methodology): “In Louppe et al. [26], the authors repurpose adversarial training to train a model that pivots on a sensitive attribute - that is, the model’s predictions are independent of the sensitive attribute.”
Invariant Risk Minimization (2019, methodology): “Third, we could adopt a domain adaptation strategy, and estimate a data representation Φ(X1, X2) that follows the same distribution for all environments [16, 33].”
Fader Networks: Manipulating Images by Sliding Attributes (2017, methodology): “This idea is inspired by the work on predictability minimization [21] and adversarial training for domain adaptation [6, 15] where the objective is also to learn an invariant latent representation using an adversarial formulation of the learning objective.”
Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics (2021, background): “In-processing techniques [6, 7, 8, 9, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33] try to remove sensitive information from the learned representation space.”
INFERNO: Inference-Aware Neural Optimisation (2018, methodology): “regressing the parameters of interest and therefore approximating the posterior mean given the data, which then can be used directly as a summary statistic. A different path is taken by Louppe et al. [24], where the authors present a adversarial training procedure to enforce a pivotal property on a ”
Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques (2020, methodology): “Thus, an alternative DeepAK8 algorithm, “DeepAK8-MD”, is developed to be largely decorrelated with the mass of a jet, while preserving the discrimination power as much as possible using an adversarial training approach [97].”
Fairness GAN (2018, background): “, considering other modalities of multimedia data in addition to images, adding the equality of odds fairness definition [24], investigating the possible advantages to allowing treatment disparity, pursuing the Gumbel-Softmax trick for discrete outcome variables [41,42], and formulating a Fairness G”
An adversarial approach for the robust classification of pneumonia from chest radiographs (2020, methodology): “ to learn a classifier that is invariant to the nuisance variable, which requires no data whatsoever from the target domain. To implement our training, we take the approach suggested in Louppe et al. [17] and adapt it for use in the application of radiograph classification. For the notation in the f”
Learning De-biased Representations with Biased Representations (2019, background): “Other approaches have proposed to remove predictability of p(B|X) based on f(X) through domain adversarial losses (Louppe et al., 2017; Wang et al., 2019b) or mutual information minimisation (Kim et al.”
Overcoming Language Priors in Visual Question Answering with Adversarial Regularization (2018, background): “Recently, other adversarial training schemes have been proposed to encourage various forms of invariance in intermediate model representations [18, 19, 27].”
A Review of Domain Adaptation without Target Labels (2019, background): “One could find a correlating word, referred to as a pivot word, that occurs frequently in both domains [13], [170].”
Fairness in Machine Learning: A Survey (2020, unknown): “Specifically for missing data, Reference [102] provides a comprehensive discussion on the considerations of missing data on fairness, Reference [55] illustrates how different methods of treating missing data have different effects on fairness, and Reference [295] discusses selection biases (and…”
Think Locally, Act Globally: Federated Learning with Local and Global Representations (2020, methodology): “To learn fair local representations, we use adversarial training (Louppe et al., 2017) to remove protected attributes (Figure 1 (d)).”
Looking Inside Jets (2019, background): “ayer Perceptron (MLP) neural network discriminant. When comparing to individual jet shape observables, such as N-subjettiness ratios or Qjet volatility, mild improvements can be achieved. 1Systematic [217,218] and theoretical [219] uncertainties can be taken into account in the performance evaluatio”
Machine learning and the physical sciences (2019, methodology): “To address this problem, an adversarial training technique was developed that ex- tends domain adaptation to domains parametrized by ν ∈ Rq (Louppe et al., 2016).”