Explaining Answers with Entailment Trees

By Bhavana Dalvi, Peter Alexander Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, Peter Clark. 2021

232 citations recorded by the source corpus.

Our goal, in the context of open-domain textual question-answering (QA), is to explain answers by showing the line of reasoning from what is known to the answer, rather than simply showing a fragment of textual evidence (a “rationale”). If this could be done, new opportunities for understanding and debugging the system’s reasoning become possible. Our approach is to generate explanations in the form of entailment trees, namely a tree of multipremise entailment steps from facts that are known, through intermediate conclusions, to the hypothesis of interest (namely the question + answer). To train a model with this skill, we created ENTAILMENTBANK, the first dataset to contain multistep entailment trees. Given a hypothesis (question + answer), we define three increasingly difficult explanation tasks: generate a valid entailment tree given (a) all relevant sentences (b) all relevant and some irrelevant sentences, or (c) a corpus. We show that a strong language model can partially solve these tasks, in particular when the relevant sentences are included in the input (e.g., 35% of trees for (a) are perfect), and with indications of generalization to other domains. This work is significant as it provides a new type of dataset (multistep entailments) and baselines, offering a new avenue for the community to generate richer, more systematic explanations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Retrieval-Augmented Generation, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yejin Choi
  2. Mohit Bansal
  3. Hannaneh Hajishirzi
  4. Luke S. Zettlemoyer
  5. Percy Liang
  6. Noah A. Smith
  7. Jianfeng Gao
  8. Christopher D. Manning

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Hannaneh Hajishirzi
  3. Trevor Darrell
  4. Yejin Choi
  5. Xiaogang Wang
  6. Dacheng Tao
  7. Ming-Hsuan Yang
  8. Jitendra Malik

In Retrieval-Augmented Generation

  1. Peter Clark
  2. Hannaneh Hajishirzi
  3. Ashish Sabharwal
  4. Tushar Khot

In Natural Language Processing

  1. Yejin Choi
  2. Hannaneh Hajishirzi
  3. Luke S. Zettlemoyer
  4. Mohit Bansal
  5. Greg Durrett
  6. Noah A. Smith
  7. Peter Clark
  8. Percy Liang

Citations in the corpus