Natural Language Deduction through Search over Statement Compositions

By Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, Greg Durrett. 2022

52 citations recorded by the source corpus.

In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence (premises) entails a hypothesis. Existing methods primarily focus on the end-to-end discriminative version of this task, but less work has treated the generative version in which a model searches over the space of statements entailed by the premises to constructively derive the hypothesis. We propose a system for doing this kind of deductive reasoning in natural language by decomposing the task into separate steps coordinated by a search procedure, producing a tree of intermediate conclusions that faithfully reflects the system's reasoning process. Our experiments on the EntailmentBank dataset (Dalvi et al., 2021) demonstrate that the proposed system can successfully prove true statements while rejecting false ones. Moreover, it produces natural language explanations with a 17% absolute higher step validity than those produced by an end-to-end T5 model.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Faithful Reasoning Using Large Language Models (2022, methodology): “Additionally, Bostrom et al. (2022) tackle the less challenging problem of determining whether an inference matches a goal state.”
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning (2022, background): “Encouraging LLMs to generate each reasoning step one at a time, for example ProofWriter (Tafjord et al., 2021; Bostrom et al., 2022), is currently the most promising direction for achieving causal reasoning, and it is the approach we take in our paper.”
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training (2022, background): “Bostrom et al. (2022) generate intermediate reasoning steps with ﬁnetuned PLMs. Zhou et al. (2022) prompt a PLM to generate subproblem descriptions for a complex problem, and feed the sub-problems back to the PLM to solve hierarchically.”
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model (2022, background): “Another line of work has focused on training language models to perform reasoning by ﬁne-tuning on gold reasoning traces (Bostrom et al., 2022; Creswell and Shanahan, 2022; Dalvi et al., 2021; Tafjord et al., 2021).”
Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering (2022, methodology): “Aside from methods that make reasoning explicit in a linear chain manner, another set of work produce recursive structures of reasoning, through either backward chaining (Dalvi et al., 2022; Jung et al., 2022) or forward chaining (Bostrom et al., 2022).”