Multi-hop Reading Comprehension through Question Decomposition and Rescoring

By Sewon Min, Victor Zhong, Luke S. Zettlemoyer, Hannaneh Hajishirzi. 2019

283 citations recorded by the source corpus.

Multi-hop Reading Comprehension (RC) requires reasoning and aggregation across several paragraphs. We propose a system for multi-hop RC that decomposes a compositional question into simpler sub-questions that can be answered by off-the-shelf single-hop RC models. Since annotations for such decomposition are expensive, we recast subquestion generation as a span prediction problem and show that our method, trained using only 400 labeled examples, generates sub-questions that are as effective as human-authored sub-questions. We also introduce a new global rescoring approach that considers each decomposition (i.e. the sub-questions and their answers) to select the best final answer, greatly improving overall performance. Our experiments on HotpotQA show that this approach achieves the state-of-the-art results, while providing explainable evidence for its decision making in the form of sub-questions.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies (2021, background): “This issue has prompted recent work to create question decompositions for existing datasets (Wolfson et al., 2020), and to train models that generate question decompositions (Perez et al., 2020; Khot et al., 2020b; Min et al., 2019).”
Augmented Language Models: a Survey (2023, methodology): “While many earlier works include learning to decompose through distant supervision (Perez et al., 2020; Talmor and Berant, 2018; Min et al., 2019), like Zhou et al. (2022), many recent works employ in-context learning to do so (Yang et al., 2022a; Khot et al., 2022; Dua et al., 2022; Kazemi et al.,…”
Break It Down: A Question Understanding Benchmark (2020, methodology): “Similar to BREAKRC, retrieved paragraphs are fed to a pretrained BERT-based RC model (Min et al., 2019b) to answer x.”
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, methodology): “6 For Natural Questions Open, we use the English Wikipedia dump from December 20, 2018, following Lee et al. (2019) and Min et al. (2019a).”
Internet-augmented language models through few-shot prompting for open-domain question answering (2022, background): “, 2021), where complex queries could be decomposed into simpler sub-queries, akin to approaches in question decomposition (Min et al., 2019; Perez et al., 2020).”
Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021, methodology): “Question Decomposition: Our work is also different from several existing works about question decomposition, where the strategy of decomposition is largely reﬂected by the question itself (Min et al., 2019; Wolfson et al., 2020).”
Unsupervised Question Decomposition for Question Answering (2020, methodology): “D ECOMP RC (Min et al., 2019b) largely frames sub-questions as extractive spans of a multi-hop question, learning to predict span-based sub-questions via supervised learning on human annotations.”
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, methodology): “On this resulting subset of 253 questions, MODULARQAis comparable to the DecompRC system with only a 2pt F1 gap.”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, background): “…either retrieve passages from a small passage pool pruned based on the the speciﬁc dataset (Sun et al., 2019; Dhingra et al., 2020), or focus on a non-retrieval setting where a compact documents set is already given (De Cao et al., 2018; Zhong et al., 2019; Tu et al., 2019; Beltagy et al., 2020).”
Hierarchical Graph Network for Multi-hop Question Answering (2019, methodology): “Then, a Machine Reading Comprehension (MRC) model is applied to the selected paragraphs for answer prediction (Nishida et al., 2019; Min et al., 2019b).”
Multi-hop Question Answering via Reasoning Chains (2019, background): “It also shows there exist other challenges to address in the question answering piece, complementary to the proposed model, like decomposing the question into different pieces (Min et al., 2019b) to further improve the multi-hop QA performance.”
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering (2020, background): “As we only consider the questions whose answers can be extracted from p , we do not add any modiﬁ-cations to the model unlike some previous studies in HotpotQA (Min et al., 2019b; Ding et al., 2019).”
Decomposed Prompting: A Modular Approach for Solving Complex Tasks (2022, methodology): “Modular Approaches for Multi-Step Reasoning Our work follows a long literature in NLP on neural modular modeling architectures (Andreas et al., 2016; Talmor & Berant, 2018; Min et al., 2019; Jiang & Bansal, 2019; Gupta et al., 2020; Perez et al., 2020; Khot et al., 2021; Levine et al., 2022) for que”
Rissanen Data Analysis: Examining Dataset Characteristics via Description Length (2021, methodology): “Yang et al. (2018) proposed HOTPOTQA as a dataset that benefits from decomposing questions into subquestions, but recent work has called the benefit into doubt (Min et al., 2019a; Jiang and Bansal, 2019; Chen and Durrett, 2019) while there is also evidence that decomposition helps (Min”
Is a Question Decomposition Unit All We Need? (2022, methodology): “Min et al. (2019) decompose questions from H OT - POT QA using span predictions based on reasoning types and picks the best decomposition using a decomposition scorer.”
F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, background): “Other approaches are based on question decomposition (Min et al., 2019; Perez et al., 2020), graph/hierarchical structures (Tu et al., 2019; Fang et al., 2019; Asai et al., 2020), virtual knowledge bases (Dhingra et al., 2020) or transformer models (Zhao et al., 2020).”
Neural Databases (2020, methodology): “In solving multi-hop questions, the works either decompose the question into simpler sub questions [27, 57], or condition each hop on the previously retrieved documents [6].”