Invariant Representations through Adversarial Forgetting
By Ayush Jaiswal, Daniel Moyer, Greg Ver Steeg, Wael AbdAlmageed, Premkumar Natarajan. 2019
47 citations recorded by the source corpus.
We propose a novel approach to achieving invariance for deep neural networks in the form of inducing amnesia to unwanted factors of data through a new adversarial forgetting mechanism. We show that the forgetting mechanism serves as an information-bottleneck, which is manipulated by the adversarial training to learn invariance to unwanted factors. Empirical results show that the proposed framework achieves state-of-the-art performance at learning invariance in both nuisance and bias settings on a diverse collection of datasets and tasks.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Aram Galstyan
- Fred Morstatter
- Ninareh Mehrabi
- Lucia Specia
- Greg Ver Steeg
- Francesca Toni
- Piyawat Lertvittayakumjorn
- Umang Gupta
In Computer Vision and Pattern Recognition
- Fred Morstatter
- Aram Galstyan
- Lucia Specia
- Greg Ver Steeg
- Umang Gupta
- Francesca Toni
- Ninareh Mehrabi
- Piyawat Lertvittayakumjorn
Citations in the corpus
- Attributing Fair Decisions with Attention Interventions (2021, methodology): “In addition to informationtheoretic approaches, we also considered baselines that use adversarial learning such as MaxEnt-ARL (Roy and Boddeti 2019), LAFTR (Madras et al. 2018), and Adversarial Forgetting (Jaiswal et al. 2020).”
- Human-in-the-loop Debugging Deep Text Classifiers (2020, background): “Adversarial training can prevent the models from exploiting irrelevant and/or protected features (Jaiswal et al., 2019; Zhang et al., 2018).”