Learning to Explain: An Information-Theoretic Perspective on Model Interpretation
By Jianbo Chen, Le Song, Martin J. Wainwright, Michael I. Jordan. 2018
666 citations recorded by the source corpus.
We introduce instancewise feature selection as a methodology for model interpretation. Our method is based on learning a function to extract a subset of features that are most informative for each given example. This feature selector is trained to maximize the mutual information between selected features and the response variable, where the conditional distribution of the response variable given the input is the model to be explained. We develop an efficient variational approximation to the mutual information, and show the effectiveness of our method on a variety of synthetic and real data sets using both quantitative metrics and human evaluation.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Information Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yoshua Bengio
- Percy Liang
- Jianfeng Gao
- Sergey Levine
- Noah A. Smith
- Ruslan Salakhutdinov
- Luke S. Zettlemoyer
In Safety, Robustness & Interpretability
- Riccardo Guidotti
- Tomas Pfister
- Sercan Ö. Arik
- Dino Pedreschi
- Yoav Goldberg
- Tommi S. Jaakkola
- Su-In Lee
- Alon Jacovi
In Information Theory
- Stefano Ermon
- Walter Karlen
- Patrick Schwab
- Tomas Pfister
- Yoav Goldberg
- Sercan Ö. Arik
- Alon Jacovi
- Sang Michael Xie
Citations in the corpus
- Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations (2019, methodology): “The methods that provide feature importance explanations [27, 23, 7, 20, 11] aim to provide the user with the subset of input tokens that contributed the most to the prediction of the model.”
- NILE : Natural Language Inference with Faithful Natural Language Explanations (2020, background): “…of a model’s predictions has been studied from different perspectives, including feature importance based explanations (Ribeiro et al., 2016; Lundberg and Lee, 2017; Chen et al., 2018), or post-hoc natural language explanations (Huk Park et al., 2018; Kim et al., 2018; Ling et al., 2017).”
- Information Constraints on Auto-Encoding Variational Bayes (2018, background): “Recent work [22, 14, 13] focuses on decomposing the second term of the ELBO and identifying terms, one of which is the total correlation between hidden variables in the aggregate posterior.”
- CXPlain: Causal Explanations for Model Interpretation under Uncertainty (2019, methodology): “In addition, the L2X method that uses a variational approximation of mutual information [31] and Bayesian nonparametrics [32] have been proposed to explain a target model.”
- Decoupling Deep Learning for Interpretable Image Recognition (2022, neutral): “The previous mask-based methods [6, 4, 32] only perform activation consistent learning for masking the same size as the original image, which is prone to adversarial effects [29].”
- Explaining by Removing: A Unified Framework for Model Explanation (2020, methodology): “However, recent work has also proposed the strategy of local feature selection (Chen et al., 2018a), and other papers have introduced methods to isolate sets of relevant features (Zhou et al., 2014; Fong and Vedaldi, 2017; Dabkowski and Gal, 2017).”
- Aligning Faithful Interpretations with their Social Attribution (2020, methodology): “Outside of NLP, Chen et al. (2018) describe select-predict models for feature-vector inputs trained to maximize mutual information between the selected features and the response variable.”
- Benchmarking and survey of explanation methods for black box models (2021, methodology): “L2X (Chen et al. 2018) is a local post-hoc model-agnostic explanation method that searches for the minimal number of pixels that change the classification.”
- Parameterized Explainer for Graph Neural Network (2020, background): “In the literature, although a variety of efforts have been undertaken to interpret general deep neural networks, existing approaches [6, 26, 12, 31, 43, 19, 20] in this line fall short in their ability to explain graph structures, which is essential for GNNs.”
- Tell me why! Explanations support learning relational and causal structure (2021, background): “That is, rather than seeking explanations post-hoc, to help humans understand the system (e.g., Chen et al., 2018; Topin & Veloso, 2019; Xie et al., 2020), these works use explanations to help the system understand the task (cf. Santoro et al., 2021).”
- Gradient Estimation with Stochastic Softmax Tricks (2020, methodology): “L2X [17] and SoftSub [86] are bespoke relaxations.”
- Explaining Knowledge Distillation by Quantifying the Knowledge (2020, neutral): “[3] proposed instance-wise feature selection via mutual information for model interpretation.”