xGEMs: Generating Examplars to Explain Black-Box Models

By Shalmali Joshi, Oluwasanmi Koyejo, Been Kim, Joydeep Ghosh. 2018

42 citations recorded by the source corpus.

This work proposes xGEMs or manifold guided exemplars, a framework to understand black-box classifier behavior by exploring the landscape of the underlying data manifold as data points cross decision boundaries. To do so, we train an unsupervised implicit generative model -- treated as a proxy to the data manifold. We summarize black-box model behavior quantitatively by perturbing data samples along the manifold. We demonstrate xGEMs' ability to detect and quantify bias in model learning and also for understanding the changes in model behavior as training progresses.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Getting a CLUE: A Method for Explaining Uncertainty Estimates (2020, background): “[22] Shalmali Joshi, Oluwasanmi Koyejo, Been Kim, and Joydeep Ghosh.”
PRover: Proof Generation for Interpretable Reasoning over Rules (2020, background): “…have focused on formalizing the notion of interpretability (Rudin, 2019; Doshi-Velez and Kim, 2017; Hase and Bansal, 2020), tweaking features for local model interpretability (Ribeiro et al., 2016, 2018) and exploring interpretability in latent spaces (Joshi et al., 2018; Samangouei et al., 2018).”
Selective Classification Can Magnify Disparities Across Groups (2020, background): “Models have been shown to latch onto spurious correlations between labels and demographic attributes such as race and gender (Buolamwini & Gebru, 2018; Joshi et al., 2018), and we study this on the CelebA dataset (Liu et al., 2015).”
Robust Feature-Level Adversaries are Interpretability Tools (2021, methodology): “Others [55, 63, 29, 28, 60, 24] have used deep generative networks, and [69] aimed to create more semantically-understandable attacks by using an autoencoder with a “disentangled” embedding space.”
Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? (2020, methodology): “Several methods exist for vision models (Joshi et al., 2018; Samangouei et al., 2018).”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, methodology): “The methods were initially proposed in the computer vision domain [67, 119], whereas Hase and Bansal [62] developed and adapted the method for text and tabular data.”