Aligning Faithful Interpretations with their Social Attribution

By Alon Jacovi, Yoav Goldberg. 2020

117 citations recorded by the source corpus.

Abstract We find that the requirement of model interpretations to be faithful is vague and incomplete. With interpretation by textual highlights as a case study, we present several failure cases. Borrowing concepts from social science, we identify that the problem is a misalignment between the causal chain of decisions (causal attribution) and the attribution of human behavior to the interpretation (social attribution). We reformulate faithfulness as an accurate attribution of causality to the model, and introduce the concept of aligned faithfulness: faithful causal chains that are aligned with their expected social behavior. The two steps of causal attribution and social attribution together complete the process of explaining behavior. With this formalization, we characterize various failures of misaligned faithful highlight interpretations, and propose an alternative causal chain to remedy the issues. Finally, we implement highlight explanations of the proposed causal format using contrastive explanations.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

ProoFVer: Natural Logic Theorem Proving for Fact Verification (2021, background): “Faithful explanations can find use cases to various stakeholders as mechanisms for dispute, debug or advice (Jacovi and Goldberg, 2021).”
Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “The vast majority of prior work that proposes models (Jain et al., 2020; Jacovi and Goldberg, 2020a) and evaluations (DeYoung et al.”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “ fails us, but this is not true for reliance because we did not attribute intent to the trustee [68]. Despite the fact that AI models are tools, people anthropomorphize them and assign intent to them [30, 52]. This makes us believe that reliance is not suitable for AI models, and for this reason we ”
Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students? (2020, background): “Such explanations, termed as “Trojan explanations”, are a problematic manifestation of several approaches, as discussed in (Chang et al., 2020; Jacovi and Goldberg, 2021).”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, background): “However, it is possible that the model learns to ignore the semantics of the explanation and instead makes predictions on the basis of the explanation’s template type (Kumar and Talukdar, 2020; Jacovi and Goldberg, 2021).”
The Unreliability of Explanations in Few-Shot In-Context Learning (2022, methodology): “Our concept of consistency resembles plausibility as described in the literature (Jacovi and Goldberg, 2021), in that we assess whether the prediction follows from the explanation as perceived by a human.”
Explaining NLP Models via Minimal Contrastive Editing (MiCE) (2020, background): “Madaan et al. (2021) applies con-such as free-text rationales (Liang et al., 2020) or highlights (Jacovi and Goldberg, 2020).”
Rationale-Inspired Natural Language Explanations with Commonsense (2021, background): “Evaluating faithfulness of NLEs is a challenging open question (Jacovi and Goldberg, 2021).”
Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations (2021, background): “For ERs, faithfulness metrics are more studied than NLEs in the literature (DeYoung et al., 2020; Jacovi & Goldberg, 2021), and both necessary and sufﬁcient conditions for faith-fulness exist.”
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning (2022, methodology): “Our concept of consistency resembles plausibility as described in Jacovi and Goldberg (2021), in that we assess whether the prediction follows from the explanation as perceived by a human.”
Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? (2020, background): “As observed by Jacovi and Goldberg (2020a), faithfulness and simulatability are closely related, but simulatability primarily captures causal attribution of explanations and not necessarily social attribution.”