Prompting Contrastive Explanations for Commonsense Reasoning Tasks

By Bhargavi Paranjape, Julian Michael, Marjan Ghazvininejad, Luke S. Zettlemoyer, Hannaneh Hajishirzi. 2021

78 citations recorded by the source corpus.

Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit. Large pretrained language models (PLMs) can achieve near-human performance on such tasks, while providing little human-interpretable evidence of the underlying reasoning they use. In this work, we show how to use these same models to generate such evidence: inspired by the contrastive nature of human explanations, we use PLMs to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer (for example, peanuts are usually salty while raisins are sweet). Conditioning model decisions on these explanations improves performance on two commonsense reasoning benchmarks, as compared to previous non-contrastive alternatives. These explanations are also judged by humans to be more relevant for solving the task, and facilitate a novel method to evaluate explanation faithfulfness.

Subfields: Natural Language Processing, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Prompting & Reasoning

Citations in the corpus

Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, background): “…explanations or clariﬁcations to improve task performance in a supervised (Rajani et al., 2019; Lampinen et al., 2022) or unsupervised (Shwartz et al., 2020) 2021) design hand-crafted templates which they use with mask-inﬁlling to produce contrastive explanations from pretrained language models.”
A Systematic Investigation of Commonsense Understanding in Large Language Models (2021, background): “…work aims to improve the zero-shot performance of language models on commonsense benchmarks, either using external commonsense knowledge bases or the output generated by the language model (Bosselut et al., 2021; Ma et al., 2021; Shwartz et al., 2020; Paranjape et al., 2021; Holtzman et al., 2021).”
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022, methodology): “Previous work along this direction mainly use manually designed prompts/templates suited for speciﬁc datasets (Paranjape et al., 2021; Mishra et al., 2021; Shwartz et al., 2020); instead, we seek to develop a general method which can learn to construct appropriate prompts automatically.”
Shepherd Pre-trained Language Models to Develop a Train of Thought: An Iterative Prompting Approach (2022, methodology): “Previous work along this direction mainly use manually designed prompts/templates suited for specific datasets (Paranjape et al., 2021; Mishra et al., 2021; Shwartz et al., 2020; Wei et al., 2022); instead, we seek to develop a general method which can learn to construct appropriate prompts…”
Reasoning with Language Model Prompting: A Survey (2022, background): “1) Single-Stage Contrastive (Paranjape et al., 2021), POTTER (Rajagopal et al.”
A Review on Language Models as Knowledge Bases (2022, background): “…the reasoning process behind their decisions using prototype networks (Schramowski et al., 2021), through highlighting fragments of the input text to justify the output (Lei et al., 2016), or using human-provided explanations in the training process (Camburu et al., 2018; Paranjape et al., 2021).”
Generated Knowledge Prompting for Commonsense Reasoning (2021, background): “Compared to prior work that elicits knowledge via clariﬁcation questions (Shwartz et al., 2020) or contrastive explanations (Paranjape et al., 2021), our approach can generate knowledge ﬂexibly, beyond the scope of pre-deﬁned templates (Table 1).”