Attentive Explanations: Justifying Decisions and Pointing to the Evidence

By Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Bernt Schiele, Trevor Darrell, Marcus Rohrbach. 2016

80 citations recorded by the source corpus.

Deep models are the defacto standard in visual decision models due to their impressive performance on a wide array of visual tasks. However, they are frequently seen as opaque and are unable to explain their decisions. In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which led to their decisions. We postulate that deep models can do this as well and propose our Pointing and Justification (PJ-X) model which can justify its decision with a sentence and point to the evidence by introspecting its decision and explanation process using an attention mechanism. Unfortunately there is no dataset available with reference explanations for visual decision making. We thus collect two datasets in two domains where it is interesting and challenging to explain decisions. First, we extend the visual question answering task to not only provide an answer but also a natural language explanation for the answer. Second, we focus on explaining human activities which is traditionally more challenging than object classification. We extensively evaluate our PJ-X model, both on the justification and pointing tasks, by comparing it to prior models and ablations using both automatic and human evaluations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Safety Verification of Deep Neural Networks (2016, background): “A recent trend towards “explainable AI” has led to approaches that learn not only how to assign the classification labels, but also additional explanations of the model, which can take the form of a justification explanation (why this decision has been reached, for example identifying the features t”
Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, background): “…explanations that might be presented to non-developer users of the systems (Hendricks et al., 2016; Kulesza et al, 2012; 2015; Lomas et al., 2012; Park et al., 2016; Ribiero et al., 2016; Rosenthal et al., 2016) This third generation is typified by many of the same approaches as the first…”
Stacking with Auxiliary Features for Visual Question Answering (2018, background): “These advantages have motivated recent work on explainable AI systems, particularly in computer vision (Antol et al., 2015; Goyal et al., 2016; Hendricks et al., 2016; Park et al., 2016).”
Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262 (2018, background): “, [59],[60]) to have the model provide human understandable explanations for its inferences.”
It Takes Two to Tango: Towards Theory of AI's Mind (2017, background): “Riberio et al. [50] explain the predictions of a classifier by learning an interpretable model locally around the prediction.”
FOIL it! Find One mismatch between Image and Language caption (2017, background): “We note that the addition of this extra step will move this work closer to the textual/visual explanation research (e.g., (Park et al., 2016; Selvaraju et al., 2016)).”
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation (2017, background): “The need in meth-ods that interpret results of opaque statistical models is widely recognised (Vellido et al., 2011; Vel-lido et al., 2012; Freitas, 2014; Li et al., 2016; Park et al., 2016).”
Explanation and Justification in Machine Learning : A Survey Or (2017, background): “[Hendricks et al. , 2016] use an LSTM caption generation model with a loss function that encourages class discriminative information to generate justiﬁcations for the image classiﬁcation of a CNN. [Park et al. , 2016] produce both a textual justiﬁcation and a visual attention map, making their appro”
Ensembling Visual Explanations for VQA (2017, background): “This has led to some work in generating explanations that help interpret the decisions made by CNNs [5, 6, 8].”
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems (2018, background): “, 2017) — or those that generate text explanations for outputs (Hendricks et al., 2016; Park et al., 2016).”
No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML (2020, unknown): “…global explanations, such as more transparent models [14, 3, 36, 58] or black-box explanations [37], and local explanations of individual algorithm decisions, which can include input evidence [40, 21], localizations [55, 45], natural language explanations [13, 19, 23], or local approximations [51].”
The Promise of Premise: Harnessing Question Premises in Visual Question Answering (2017, background): “Some other recent work has developed models which produce natural language explanations for their outputs (Park et al., 2016; Wang et al., 2016), but there has not been work on generating explanations for irrelevant questions or false premises.”
Understanding Attention and Generalization in Graph Neural Networks (2019, methodology): “The practical importance of attention in deep learning is well-established and there are many arguments in its favor [1], including interpretability [2, 3].”
Explainable Neural Computation via Stack Neural Module Networks (2018, background): “An alternative line of work investigates the extent to which models can be explicitly designed from the outset to provide enhanced interpretability, where main focus of study has been visual attention [22,25].”
Recent Advances and Trends in Multimodal Deep Learning: A Review (2021, unknown): “0 [160], VQA-CP [6], VQA-X [64], Ok-VQA [95], and FVQA [151].”
A historical perspective of explainable Artificial Intelligence (2020, background): “…the latter can be tracked in terms of interpretable mechanisms, for example, (Craven & Shaolin, 1995; Frost & Hinton, 2017 (Hendricks et al., 2016; Park et al., 2016; Same, Wigand, & Müller, 2019) or groups of input – output tokens that are causally related, for example, (Alvarez-Melisa &…”