Visual Feature Attribution Using Wasserstein GANs

By Christian F. Baumgartner, Lisa M. Koch, Kerem Can Tezcan, Jia Xi Ang, Ender Konukoglu. 2017

158 citations recorded by the source corpus.

Attributing the pixels of an input image to a certain category is an important and well-studied problem in computer vision, with applications ranging from weakly supervised localisation to understanding hidden effects in the data. In recent years, approaches based on interpreting a previously trained neural network classifier have become the de facto state-of-the-art and are commonly used on medical as well as natural image datasets. In this paper, we discuss a limitation of these approaches which may lead to only a subset of the category specific features being detected. To address this problem we develop a novel feature attribution technique based on Wasserstein Generative Adversarial Networks (WGAN), which does not suffer from this limitation. We show that our proposed method performs substantially better than the state-of-the-art for visual attribution on a synthetic dataset and on real 3D neuroimaging data from patients with mild cognitive impairment (MCI) and Alzheimer's disease (AD). For AD patients the method produces compellingly realistic disease effect maps which are very close to the observed effects.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Generative Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

In Generative Models

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “This generative framework has been adopted by several recent studies [366,367] mainly as an attribution method to relate a particular output of a Deep Learning model to their input variables.”
Generative Adversarial Network in Medical Imaging: A Review (2018, background): “, 2017c), highlighting regions most accountable for a disease (Baumgartner et al., 2017) and re-colorization of endoscopic video data (Ross et al.”
Causability and explainability of artificial intelligence in medicine (2019, background): “Baumgartner, Koch, Tezcan, Ang, and Konukoglu (2017) demonstrate how image areas that are specific to the foreground class in Wasserstein Generative Adversarial Networks (WGAN) can be identified and high-lighted in the data.”
Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities (2021, background): “Two recent research directions have been highlighted in this context [2]: using generative models as an attribution method to show a direct relationship between a particular output and its input variables [114].”
Accurate brain age prediction with lightweight deep neural networks (2019, background): “In neuroimaging, deep learning has had some successes, and yet faces several challenges (Arslan et al., 2018; Baumgartner et al., 2018; Cole et al., 2017; Kawahara et al., 2017).”
GANs for Medical Image Analysis (2018, methodology): “Inspired by the AnoGAN, [9] proposes the visual attribution GAN (VA-GAN) for Alz-heimer ’ s disease detection.”
Diffusion Models for Medical Anomaly Detection (2022, background): “Other approaches focus on Generative Adversarial Networks (GANs) [9] for image-to-image translation [24,5,27].”
Reconstructing subject‐specific effect maps (2017, unknown): “Recent registrationfree methods for identifying visual attributes is an interesting direction for removing the need for registration (Baumgartner et al., 2018).”
Automatic Shadow Detection in 2D Ultrasound (2018, methodology): “Inspired by [5], we propose a novel shadow detection model to predict a dense, anatomically agnostic shadow-confidence map using only weak annotations.”