Explain Yourself! Leveraging Language Models for Commonsense Reasoning

By Nazneen Rajani, Bryan McCann, Caiming Xiong, Richard Socher. 2019

649 citations recorded by the source corpus.

Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of world-knowledge or reasoning over information not immediately present in the input. We collect human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations (CoS-E). We use CoS-E to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation (CAGE) framework. CAGE improves the state-of-the-art by 10% on the challenging CommonsenseQA task. We further study commonsense reasoning in DNNs using both human and auto-generated explanations including transfer to out-of-domain tasks. Empirical results indicate that we can effectively leverage language models for commonsense reasoning.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “To endow models with the ability to produce intermediate steps, prior work typically ﬁnetunes models on either manually annotated training datasets (Camburu et al., 2018; Rajani et al., 2019, inter alia ) or generates synthetic datasets (Talmor et al., 2020; Zelikman et al., 2022).”
Scaling Instruction-Finetuned Language Models (2022, background): “, 2022, inter alia) or focuses on models of substantially smaller scale (Ling et al., 2017; Camburu et al., 2018; Rajani et al., 2019; Talmor et al., 2020; Zelikman et al., 2022).”
ExpBERT: Representation Engineering with Natural Language Explanations (2020, background): “Rather than using language to define a global collection of features, Rajani et al. (2019) and Camburu et al. (2018) use instance-level explanations to train models that generate their own explanations.”
Shaping Visual Representations with Language for Few-Shot Classification (2019, neutral): “…interpretability for both textual (e.g. natural language inference; [3]) and visual [17, 18] tasks, but here we examine whether this act of predicting language can actually improve downstream task performance; similar ideas have been explored in text [22] and reinforcement learning [2, 14] domains.”
Recursively Summarizing Books with Human Feedback (2021, background): “While Dayan and Hinton (1993) and Christiano et al. (2018) only tried this on simple algorithmic tasks, Perez et al. (2020) and Rajani et al. (2019) use similar ideas for question-answering tasks using a single step of decomposition.”
GeDi: Generative Discriminator Guided Sequence Generation (2020, background): “Methods for controlling text generation can be categorized broadly into two types: training or finetuning a model directly for controllable generation (Keskar et al., 2019; Ziegler et al., 2019; Rajani et al., 2019; Ficler & Goldberg, 2017; Yu et al., 2017; Hu et al., 2017) or using a discriminator ”
Large Language Models are Zero-Shot Reasoners (2022, background): “…et al., 2022, Rae et al., 2021], but its ability can be substantially increased by making them produce step-by-step reasoning, either by ﬁne-tuning [Rajani et al., 2019, Cobbe et al., 2021, Zelikman et al., 2022, Nye et al., 2022] or few-shot prompting [Wei et al., 2022, Wang et al., 2022,…”
Can language models learn from explanations in context? (2022, methodology): “…explored training or tuning of language processing models with task instructions (e.g. Raffel et al., 2020; Wei et al., 2021; Ouyang et al., 2022; Sanh et al., 2021), and with explanations (Cam-buru et al., 2018; Rajani et al., 2019; Narang et al., 2020; Zhou et al., 2020; Hase and Bansal, 2021).”
STaR: Bootstrapping Reasoning With Reasoning (2022, methodology): “Many have pointed out that CQA contains a number of biases, along several dimensions including gender [3].”
Reframing Human-AI Collaboration for Generating Free-Text Explanations (2021, methodology): “11 (Rajani et al., 2019) and ECQA (Aggarwal et al., 2021) datasets.”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, background): “…time and generate such explanations for their decisions at deployment time (Hendricks et al., 2016; Ling et al., 2017; Park et al., 2018; Camburu et al., 2018; Kim et al., 2018; Rajani et al., 2019; Camburu et al., 2020; Narang et al., 2020; Kumar and Taluk-dar, 2020; Marasovi´c et al., 2022).”
Few-Shot Self-Rationalization with Natural Language Prompts (2021, methodology): “We exclude C O S-E (Rajani et al., 2019) as it is too noisy to be useful for modeling and evaluating self-rationalization (Narang et al., 2020), but we do not support using C O S-E in the future, especially since ECQA is introduced.”
Flexible Generation of Natural Language Deductions (2021, background): “Modeling the reasoning process in this way would grant the ability to generalize systematically to any problem that could be decomposed in terms of available operations, among other desirable properties (Rudin, 2019).”
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data (2021, background): “2 (Camburu et al., 2018; Rajani et al., 2019; Kumar & Talukdar, 2020; Hase et al., 2020; Wiegreffe et al., 2020).”
Measuring Association Between Labels and Free-Text Rationales (2020, background): “Here, extractive rationales necessarily fall short—rationales must instead take the form of free-text natural language to fill in the reasoning or knowledge gap (Camburu et al., 2018; Rajani et al., 2019).”