Learning to Explain With Complemental Examples

By Atsushi Kanehira, Tatsuya Harada. 2018

44 citations recorded by the source corpus.

This paper addresses the generation of explanations with visual examples. Given an input sample, we build a system that not only classifies it to a specific category, but also outputs linguistic explanations and a set of visual examples that render the decision interpretable. Focusing especially on the complementarity of the multimodal information, i.e., linguistic and visual examples, we attempt to achieve it by maximizing the interaction information, which provides a natural definition of complementarity from an information theoretical viewpoint. We propose a novel framework to generate complemental explanations, on which the joint distribution of the variables to explain, and those to be explained is parameterized by three different neural networks: predictor, linguistic explainer, and example selector. Explanation models are trained collaboratively to maximize the interaction information to ensure the generated explanation are complemental to each other for the target. The results of experiments conducted on several datasets demonstrate the effectiveness of the proposed method.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “11 shows an example for illustrating how to work the proposal of [159].”
Making deep neural networks right for the right scientific reasons by interacting with their explanations (2020, background): “Lastly, because it is not yet clear what makes explanations good for humans [35], one should extend explanatory interactions towards using alternative explanations, multiple modalities and counterfactuals [36, 37].”
Machine Learning Interpretability: A Survey on Methods and Metrics (2019, methodology): “There are CVPR 2019 papers with a great focus on interpretability and explainability for computer vision, such as explainability methods for graph CNNs [138], interpretable and fine-grained visual explanations for CNNs [139], interpreting CNNs via Decision Trees [140], and learning to explain with c”
Multimodal Explanations by Predicting Counterfactuality in Videos (2018, background): “Methods exploiting textual [14] or multimodal [20, 2, 17] supervision belong to this category.”
Generative causal explanations of black-box classifiers (2020, methodology): “It is interesting to note that this metric, derived in our work from ﬁrst principles of causality using the DAG in Figure 1(b), has been used as a heuristic for selecting informative features in other work on explanation [56, 11, 40, 57–59].”