Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction

By Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, Junji Tomita. 2019

109 citations recorded by the source corpus.

Question answering (QA) using textual sources for purposes such as reading comprehension (RC) has attracted much attention. This study focuses on the task of explainable multi-hop QA, which requires the system to return the answer with evidence sentences by reasoning and gathering disjoint pieces of the reference texts. It proposes the Query Focused Extractor (QFE) model for evidence extraction and uses multi-task learning with the QA model. QFE is inspired by extractive summarization models; compared with the existing method, which extracts each evidence sentence independently, it sequentially extracts evidence sentences by using an RNN with an attention mechanism on the question sentence. It enables QFE to consider the dependency among the evidence sentences and cover important information in the question sentence. Experimental results show that QFE with a simple RC baseline model achieves a state-of-the-art evidence extraction score on HotpotQA. Although designed for RC, it also achieves a state-of-the-art evidence extraction score on FEVER, which is a recognizing textual entailment task on a large textual database.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

UnifiedQA: Crossing Format Boundaries With a Single QA System (2020, methodology): “For the AB format, we use ROUGE-L metric (Lin et al., 2006; Min et al., 2019; Nishida et al., 2019).”
Hierarchical Graph Network for Multi-hop Question Answering (2019, methodology): “…the beginning and end points of the ground-truth span : (Min et al., 2019b) 55.20 69.63 - - - - ChainEx (Chen et al., 2019) 61.20 74.11 - - - - Baseline Model (Yang et al., 2018) 45.60 59.02 20.32 64.49 10.83 40.16 QFE (Nishida et al., 2019) 53 times, and n is the number of tokens for the context.”
Multi-hop Question Answering via Reasoning Chains (2019, neutral): “DecompRC (Min et al., 2019b) 55.20 69.63 N QFE (Nishida et al., 2019) 53.86 68.06 Y DFGN (Qiu et al., 2019) 56.31 69.69 Y HGN (Fang et al., 2019) 66.07 79.36 Y SAE (Tu et al., 2019a) 66”
F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, methodology): “…built upon the model by Qi et al. (2019) as it is an improved version of the BiDaf++ model, which is used in numerous state-of-the-art XQA models (Yang et al., 2018; Qi et al., 2019; Nishida et al., 2019; Ye et al., 2019; Qiu et al., 2019) including the best-scoring publication (Shao et al., 2020).”
Natural Language Deduction through Search over Statement Compositions (2022, background): “Multi-hop question answering models (Chen et al. 2019; Min et al. 2019; Nishida et al. 2019, inter alia) also build derivations linking multiple statements to support a conclusion.”
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models (2021, background): “…a model in natural language processing to perform multiple tasks is to have a shared pre-trained language model (LM), and add a small, compact neural network, often termed head, on top of the LM, for each task (Clark et al., 2019; Liu et al., 2019b; Nishida et al., 2019; Hu and Singh, 2021).”
Is Graph Structure Necessary for Multi-hop Question Answering? (2020, unknown): “Joint EM Joint F1 Baseline (Yang et al., 2018) 10.83 40.16 QFE (Nishida et al., 2019) 34.63 59.61 DFGN (Qiu et al., 2019) 33.62 59.82 TAP2 (Glass et al., 2019) 39.77 69.12 HGN (Fang et al., 2019) 43.57 71.03 SAE (Tu et al., 2019a) 45 eral extra modules in the graph fusion block, including…”
Answering Any-hop Open-domain Questions with Iterative Document Reranking (2020, methodology): “Following the existing studies [1, 23, 24], we evaluate the paragraph-level retrieval accuracy using the Paragraph Exact Match (EM) metric, which compares the top 2 paragraphs with the gold supporting paragraphs.”
T RANSFORMER -XH: M ULTI - HOP Q UESTION A N - SWERING WITH E X TRA H OP A TTENTION (2019, unknown): “Various solutions have been proposed for Hotpot QA (Min et al., 2019b; Feldman & El-Yaniv, 2019; Nishida et al., 2019).”
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks (2021, methodology): “Unlike prior multi-task learning work in QA relying on available annotated data (Lee et al., 2021; Nishida et al., 2019) or heuristics such as answer string matching to label pseudo evidential-ity (Fajcik et al., 2021), our approach is applicable to diverse downstream tasks, where we cannot use…”
A Simple Yet Strong Pipeline for HotpotQA (2020, methodology): “While Nishida et al. (2019) also use a standard Reading Comprehension (RC) model, they combine it with a special Query Focused Extrac-tor ( QFE ) module to select relevant sentences for QA and explanation.”
Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents (2019, methodology): “The ﬁrst direction focuses on applying or adapting previous techniques that are successful in single-document QA tasks to multi-document QA tasks, for example the studies in (Dhingra et al. 2018; Zhong et al. 2019; Yang et al. 2018; Nishida et al. 2019).”
Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention (2020, methodology): “We include other approaches on the FullWiki setting: Ofﬁcial Baseline (Yang et al., 2018), MUPPET (Feldman & El-Yaniv, 2019), QFE (Nishida et al., 2019), and DecompRC (Min et al., 2019a), Implementation Details.”
DDRQA: Dynamic Document Reranking for Open-domain Multi-hop Question Answering (2020, methodology): “Following existing studies (Asai et al., 2020; Nie et al., 2019; Nishida et al., 2019), we evaluate the paragraph-level retrieval accuracy using the Paragraph Exact Match (EM) metric, which compares the top 2 paragraphs with the gold supporting paragraphs.”