Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering

By Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, Caiming Xiong. 2019

327 citations recorded by the source corpus.

Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question. This paper introduces a new graph-based recurrent retrieval approach that learns to retrieve reasoning paths over the Wikipedia graph to answer multi-hop open-domain questions. Our retriever model trains a recurrent neural network that learns to sequentially retrieve evidence paragraphs in the reasoning path by conditioning on the previously retrieved documents. Our reader model ranks the reasoning paths and extracts the answer span included in the best reasoning path. Experimental results show state-of-the-art results in three open-domain QA datasets, showcasing the effectiveness and robustness of our method. Notably, our method achieves significant improvement in HotpotQA, outperforming the previous best model by more than 14 points.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, methodology): “3 Model NaturalQuestions TriviaQA SQuAD Open DrQA (Chen et al., 2017) - - - 29.8 Multi-Passage BERT (Wang et al., 2019) - - - 53.0 Path Retriever (Asai et al., 2020) 31.7 - - 56.5 Graph Retriever (Min et al., 2019b) 34.7 55.8 - - Hard EM (Min et al., 2019a) 28.8 50.9 - - ORQA (Lee et al., 2019)…”
REALM: Retrieval-Augmented Language Model Pre-Training (2020, methodology): “Approaches such as DrQA (Chen et al., 2017), HardEM (Min et al., 2019a), GraphRetriever (Min et al., 2019b), and PathRetriever (Asai et al., 2019) in Table 1 are in this category.”
How Much Knowledge Can You Pack into the Parameters of a Language Model? (2020, methodology): “by (Lee et al., 2019; Min et al., 2019b,a; Asai et al., 2019) where the model is only required to produce a single normalized answer and 2) the standard multi-answer variant used with reading comprehension systems”
Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, background): “…reasoning paths, such as constructing semantic graphs (Xu et al., 2021a), learning an RNN to retrieve reasoning paths over the Wikipedia graph (Asai et al., 2020), ﬁne-tuning with human annotated reasoning paths on math problems (Cobbe et al., 2021), or training an extractor with…”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, neutral): “To the best Method Dataset NaturalQuestions-Open WebQuestions CuratedTREC DrQA (Chen et al., 2017) - 20.7 25.7 R 3 (Wang et al., 2017) - 17.1 28.4 DSQA (Lin et al., 2018) - 25.6 29.1 HardEM (Min et al., 2019a) 28.1 - - PathRetriever (Asai et al., 2019) 32.6 - - WKLM (Xiong et al., from methods that ”
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, methodology): “, 2019) or hyperlinks (Asai et al., 2020).”
AmbigQA: Answering Ambiguous Open-domain Questions (2020, methodology): “Although Min et al. (2019b) outperforms Asai et al. (2020) on NQ-OPEN , it is worse on A MBIG NQ.”
ProoFVer: Natural Logic Theorem Proving for Fact Verification (2021, background): “Use of hyperlink structure in Wikipedia or performing entity linking to establish hyperlinked mentions, similar to our approach here, has been previously explored in multi-hop open domain question answering (Asai et al., 2020; Nie et al., 2019).”
Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “These recent neural approaches over unstructured text have overtaken prior methods exploiting the graph structure of knowledge sources (such as hyperlinks in Wikipedia) (Min et al., 2019; Asai et al., 2020; Sun et al., 2019; Xiong et al., 2019), and are an attractive alternative for dialogue.”
Learning Dense Representations of Phrases at Scale (2020, methodology): “For Natural Questions (Kwiatkowski et al., 2019), we use the pre-processed version provided by Asai et al. (2020).”
Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021, methodology): “, 2019) have been successfully used in multi-hop reasoning problems (Banerjee et al., 2020; Asai et al., 2019; Yadav et al., 2019).”
Rationale-Augmented Ensembles in Language Models (2022, methodology): “One relevant line of work tries to improve ratio-nalization and interpretability in natural language processing models, for example, by extracting rationales using task-speciﬁc approaches (Xu et al., 2021; Asai et al., 2020; Chen et al., 2019).”
Adaptive Information Seeking for Open-Domain Question Answering (2021, methodology): “, 2021)), or using the mixed but fixed strategy (strategy 3 (Asai et al., 2020)) cannot answer the”
Relevance-guided Supervision for OpenQA with ColBERT (2020, background): “, 2019b) and PathRetriever (Asai et al., 2020) propose graph-based augmentation mechanisms to classical bag-of-words IR models (in particular, BM25 and TF-IDF).”
Self-Supervised Learning: Generative or Contrastive (2020, background): “DEEP neural networks [71] have shown outstanding performance on various machine learning tasks, especially on supervised learning in computer vision (image classification [26], [48], [54], semantic segmentation [39], [76]), natural language processing (pre-trained language models [27], [68], [75], [”
Generation-Augmented Retrieval for Open-Domain Question Answering (2020, methodology): “For passage reading, we compare with both extractive (Min et al., 2019a; Asai et al., 2019; Lee et al., 2019; Min et al., 2019b; Guu et al., 2020; Karpukhin et al., 2020) and generative (Brown et al., 2020; Roberts et al., 2020; Min et al., 2020; Lewis et al., 2020a; Izacard and Grave, 2020)…”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, methodology): “After we rerank the top-100 sequences from the dense retriever, our passage recall is better than the state-of-the-art Graph Recurrent Retriever, which uses BERT to process more than 500 passages.”
Differentiable Reasoning over a Virtual Knowledge Base (2020, neutral): “86 Rec Ret (Asai et al., 2020) 500∗ 133.”