Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations

By Oana-Maria Camburu, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz, Phil Blunsom. 2019

105 citations recorded by the source corpus.

To increase trust in artificial intelligence systems, a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions. In this work, we show that such models are nonetheless prone to generating mutually inconsistent explanations, such as ”Because there is a dog in the image.” and ”Because there is no dog in the [same] image.”, exposing flaws in either the decision-making process of the model or in the generation of the explanations. We introduce a simple yet effective adversarial framework for sanity checking models against the generation of inconsistent natural language explanations. Moreover, as part of the framework, we address the problem of adversarial attacks with full target sequences, a scenario that was not previously addressed in sequence-to-sequence attacks. Finally, we apply our framework on a state-of-the-art neural natural language inference model that provides natural language explanations for its predictions. Our framework shows that this model is capable of generating a significant number of inconsistent explanations.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, background): “Some prior work has shown that language models can suffer from inconsistency in conversation (Adiwardana et al., 2020), explanation generation (Camburu et al., 2020), and factual knowledge extraction (Elazar et al., 2021).”
Scaling Instruction-Finetuned Language Models (2022, background): “, 2021), and natural language inference (Camburu et al., 2020).”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, background): “…time and generate such explanations for their decisions at deployment time (Hendricks et al., 2016; Ling et al., 2017; Park et al., 2018; Camburu et al., 2018; Kim et al., 2018; Rajani et al., 2019; Camburu et al., 2020; Narang et al., 2020; Kumar and Taluk-dar, 2020; Marasovi´c et al., 2022).”
Measuring and Improving Consistency in Pretrained Language Models (2021, methodology): “Li et al. (2019) measure and mitigate inconsistency in natural language inference (NLI), and ﬁnally, Camburu et al. (2020) propose a method for measuring inconsistencies in NLI explanations (Camburu et al., 2018).”
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks (2021, background): “First adoptions of NLEs have been in image classification [23] and were further extended to self-driving cars [26], visual question-answering [37], and natural language processing [11; 39; 9; 6; 27; 35; 28; 12].”
NILE : Natural Language Inference with Faithful Natural Language Explanations (2020, background): “Further analysis of the free-form human-provided explanations has revealed clear differences in the form of explanations, through alignment to label-speciﬁc templates (Camburu et al., 2018; Oana-Maria et al., 2019a).”
Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations (2021, background): “However, machine-generated NLEs, like other generated text, are prone to lacking background knowledge (e.g., commonsense) (Camburu et al., 2020; Mao et al., 2019).”
Rationale-Inspired Natural Language Explanations with Commonsense (2021, background): “However, machine-generated NLEs, like other generated text, are prone to lacking commonsense (Camburu et al., 2020; Mao et al., 2019).”
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (2021, background): “[21] report: “Explanations in E-SNLI largely follow a set of label-specific templates.”
LIREx: Augmenting Language Inference with Relevant Explanation (2020, background): “Issue 2: Inclusion of Spurious Explanations NLEs that are inconsistent with commonsense logic provide little help for model prediction (Camburu et al. 2020).”
Teach Me to Explain: A Review of Datasets for Explainable NLP (2021, background): “This turned out to be the case with natural language inference (Camburu et al., 2020, also re-ported in Brahman et al., 2021): Explanations in E-SNLI largely follow a set of label-speciﬁc templates.”
A Review on Language Models as Knowledge Bases (2022, background): “Camburu et al. (2020) show that by using an adversarial framework to interject modiﬁed inputs, they are able to show that LMs generate a large number of inconsistencies in their explanations.”
Prompting Contrastive Explanations for Commonsense Reasoning Tasks (2021, background): “…like SNLI (Camburu et al., 2018), Winograd Schemas (Zhang et al., 2020) and CommonsenseQA (Rajani et al., 2019b) through explicit human supervision, which are inherently ambiguous, incomplete and consequently, expensive to collect and evaluate on (Camburu et al., 2019b,a; DeYoung et al., 2020).”