On the Global Optima of Kernelized Adversarial Representation Learning
By Bashir Sadeghi, Vishnu Naresh Boddeti. 2019
36 citations recorded by the source corpus.
Adversarial representation learning is a promising paradigm for obtaining data representations that are invariant to certain sensitive attributes while retaining the information necessary for predicting target attributes. Existing approaches solve this problem through iterative adversarial minimax optimization and lack theoretical guarantees. In this paper, we first study the ``linear" form of this problem i.e., the setting where all the players are linear functions. We show that the resulting optimization problem is both non-convex and non-differentiable. We obtain an exact closed-form expression for its global optima through spectral learning and provide performance guarantees in terms of analytical bounds on the achievable utility and invariance. We then extend this solution and analysis to non-linear functions through kernel representation. Numerical experiments on UCI, Extended Yale B and CIFAR-100 datasets indicate that, (a) practically, our solution is ideal for ``imparting" provable invariance to any biased pre-trained data representation, and (b) the global optima of the ``kernel" form can provide a comparable trade-off between utility and invariance in comparison to iterative minimax optimization of existing deep neural network based approaches, but with provable guarantees.
Subfields: Artificial Intelligence, Computational Mechanics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Praneeth Vepakomma
- Ramesh Raskar
- Abhishek Singh
- Otkrist Gupta
- Ehsan Adeli
- Kilian M. Pohl
- Qingyu Zhao
- Chaoyang He
In Computational Mechanics
- Otkrist Gupta
- Ramesh Raskar
- Abhishek Singh
- Ehsan Adeli
- Kilian M. Pohl
- Praneeth Vepakomma
- Qingyu Zhao
Citations in the corpus
- NoPeek: Information leakage reduction to share activations in distributed deep learning (2020, background): “A kernelized version of such an adversarial learning approach with theoretical guarantees is provided in [36].”
- Training confounder-free deep learning models for medical applications (2020, methodology): “[7] proposed to use the MSE loss between the predicted and ground-truth confounder value as the adversarial loss for invariant feature learning.”