Repurposing Entailment for Multi-Hop Question Answering Tasks

By Harsh Trivedi, Heeyoung Kwon, Tushar Khot, Ashish Sabharwal, Niranjan Balasubramanian. 2019

50 citations recorded by the source corpus.

Question Answering (QA) naturally reduces to an entailment problem, namely, verifying whether some text entails the answer to a question. However, for multi-hop QA tasks, which require reasoning with multiple sentences, it remains unclear how best to utilize entailment models pre-trained on large scale datasets such as SNLI, which are based on sentence pairs. We introduce Multee, a general architecture that can effectively use entailment models for multi-hop QA tasks. Multee uses (i) a local module that helps locate important sentences, thereby avoiding distracting information, and (ii) a global module that aggregates information by effectively incorporating importance weights. Importantly, we show that both modules can use entailment functions pre-trained on a large scale NLI datasets. We evaluate performance on MultiRC and OpenBookQA, two multihop QA datasets. When using an entailment function pre-trained on NLI datasets, Multee outperforms QA models trained only on the target QA datasets and the OpenAI transformer models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Multi-hop Question Answering via Reasoning Chains (2019, background): “This represents a step towards building models that represent the reasoning process more explicitly (Trivedi et al., 2019; Jiang et al., 2019).”
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project (2019, background): “Details of how this is done are given in (Trivedi et al., 2019).”
Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization (2021, methodology): “Trivedi et al. (2019) use hand-crafted rules to ﬁrst cast MCRC to NLI; and subsequently divide a long passage into smaller sentence-level premises.”
OCNLI: Original Chinese Natural Language Inference (2020, background): “…representation and transfer learning (Conneau et al., 2017; Subrama-nian et al., 2018; Reimers and Gurevych, 2019), automated question-answering (Khot et al., 2018; Trivedi et al., 2019) and model probing (Warstadt et al., 2019; Richardson et al., 2020; Geiger et al., 2020; Jeretic et al., 2020).”
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks (2021, unknown): “Using entailment models to enhance the performance of QA tasks has been extensively studied (Harabagiu and Hickl, 2006; Sacaleanu et al., 2008; Abacha and Demner-Fushman, 2019; Trivedi et al., 2019).”
Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options (2020, methodology): “These datasets are useful in three settings: evaluation (Williams et al., 2018; Rajpurkar et al., 2018; Zellers et al., 2019); pretraining (Phang et al., 2018; Conneau et al., 2018; Pruksachatkun et al., 2020); and as training data for downstream tasks (Trivedi et al., 2019; Portelli et al., 2020).”
Can NLI Models Verify QA Systems' Predictions? (2021, background): “For question answering, Harabagiu and Hickl (2006) showed that textual entailment can be used to enhance the accuracy of the open-domain QA systems; Trivedi et al. (2019) used a pretrained NLI model to select relevant sentences for multi-hop question answering; Yin et al. (2020) tested whether NLI…”
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, methodology): “Although various neural QA methods have achieved high performance on some of these datasets (Sun et al., 2018; Trivedi et al., 2019; Tymoshenko et al., 2017; Seo et al., 2016; Wang and Jiang, 2016; De Cao et al., 2018; Back et al., 2018), we argue that more effort must be dedicated to explaining…”
MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark (2024, unknown): “A A PPENDIX Evaluation Strategy To evaluate ALE on MMAU, we adopt methods similar to those used for assessing question-response performance in entailment models (Deshmukh et al., 2024; Trivedi et al., 2019).”