Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

By Sewon Min, Danqi Chen, Luke S. Zettlemoyer, Hannaneh Hajishirzi. 2019

112 citations recorded by the source corpus.

We introduce an approach for open-domain question answering (QA) that retrieves and reads a passage graph, where vertices are passages of text and edges represent relationships that are derived from an external knowledge base or co-occurrence in the same article. Our goals are to boost coverage by using knowledge-guided retrieval to find more relevant passages than text-matching methods, and to improve accuracy by allowing for better knowledge-guided fusion of information across related passages. Our graph retrieval method expands a set of seed keyword-retrieved passages by traversing the graph structure of the knowledge base. Our reader extends a BERT-based architecture and updates passage representations by propagating information from related passages and their relations, instead of reading each passage in isolation. Experiments on three open-domain QA datasets, WebQuestions, Natural Questions and TriviaQA, show improved performance over non-graph baselines by 2-11% absolute. Our approach also matches or exceeds the state-of-the-art in every case, without using an expensive end-to-end training regime.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “Augmenting text-based retrieval with external structured information, such as knowledge graph and Wikipedia hyperlinks, has also been explored recently (Min et al., 2019b; Asai et al., 2020).”
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, methodology): “…(Chen et al., 2017) - - - 29.8 Multi-Passage BERT (Wang et al., 2019) - - - 53.0 Path Retriever (Asai et al., 2020) 31.7 - - 56.5 Graph Retriever (Min et al., 2019b) 34.7 55.8 - - Hard EM (Min et al., 2019a) 28.8 50.9 - - ORQA (Lee et al., 2019) 31.3 45.1 - 20.2 REALM (Guu et al., 2020) 38.2 - -…”
REALM: Retrieval-Augmented Language Model Pre-Training (2020, methodology): “Approaches such as DrQA (Chen et al., 2017), HardEM (Min et al., 2019a), GraphRetriever (Min et al., 2019b), and PathRetriever (Asai et al., 2019) in Table 1 are in this category.”
How Much Knowledge Can You Pack into the Parameters of a Language Model? (2020, background): “After fine-tuning, T5-11B + SSM achieves a recall of 36.2 on the validation set, which lags behind the state-of-theart score of 51.9 from Pan et al. (2019)4 but outperforms the best baseline published alongside the dataset (recall of 33.2 (Kwiatkowski et al., 2019)).”
Entities as Experts: Sparse Memory Access with Entity Supervision (2020, neutral): “We compare our system to two paradigms for QA: Open-Book systems (ORQA and Graph Retriever), which ﬁrst retrieve a set of documents from a large corpus with the question and predict a span in the retrieved document as an answer and Closed-Book system ( RELIC and T5): model answers the question from ”
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2020, methodology): “To the best Method Dataset NaturalQuestions-Open WebQuestions CuratedTREC DrQA (Chen et al., 2017) - 20.7 25.7 R 3 (Wang et al., 2017) - 17.1 28.4 DSQA (Lin et al., 2018) - 25.6 29.1 HardEM (Min et al., 2019a) 28.1 - - PathRetriever (Asai et al., 2019) 32.6 - - WKLM (Xiong et al., from methods that ”
AmbigQA: Answering Ambiguous Open-domain Questions (2020, methodology): “Although Min et al. (2019b) outperforms Asai et al. (2020) on NQ-OPEN , it is worse on A MBIG NQ.”
Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “These recent neural approaches over unstructured text have overtaken prior methods exploiting the graph structure of knowledge sources (such as hyperlinks in Wikipedia) (Min et al., 2019; Asai et al., 2020; Sun et al., 2019; Xiong et al., 2019), and are an attractive alternative for dialogue.”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, background): “Both BM25-uni and dense retrieval models peak at document collection passage length 200 and their combination outperforms the best prior pipeline model (Min et al., 2019).”
Relevance-guided Supervision for OpenQA with ColBERT (2020, background): “Both GraphRetriever (Min et al., 2019b) and PathRetriever (Asai et al.”
Generation-Augmented Retrieval for Open-Domain Question Answering (2020, methodology): “Many extractive QA methods (Chen et al., 2017; Min et al., 2019b; Guu et al., 2020; Karpukhin et al., 2020) measure the probability of span extraction in different retrieved passages independently, despite that their collective signals may provide more evidence in determining the correct answer.”
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2020, neutral): “Besides, Model EM BM25+BERT (Lee et al., 2019) 26.5 HardEM (Min et al., 2019a) 28.1 GraphRetriever (Min et al., 2019b) 34.5 PathRetriever (Asai et al., 2020) 32.6 ORQA (Lee et al., 2019) 33.3 REALM (Guu et al., 2020) 40.4 DPR (Karpukhin et al., 2020) 41.5 GAR (Mao et al., 2020) 41.6 RocketQA + DPR r”
Phrase Retrieval Learns Passage Retrieval, Too (2021, background): “There has also been a line of work to integrate text retrieval with structured knowledge graphs (Sun et al., 2018, 2019; Min et al., 2020).”
Open-Domain Question Answering (2020, methodology): “The approaches include (1) how to leverage structured data to guide the retriever or reader stage of existing textual QA systems (Asai et al., 2020; Min et al., 2019b), or (2) how to synthesize information from these two heterogeneous sources and build effective QA models on the combined information”
Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning (2020, background): “approaches (Ding et al., 2019; Godbole et al., 2019; Min et al., 2019b; Asai et al., 2019), where paragraphs are connected if they share the same-named entities.”
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering (2021, methodology): “…2020); (2) reranking-based methods: RIDER (Mao et al., 2021) and RECONSIDER (Iyer et al., 2021); (3) leveraging knowledge graphs or graph information between passages: Graph-Retriever (Min et al., 2019), PathRetriever (Asai et al., 2020), KAQA (Zhou et al., 2020), and UniK-QA (Oguz et al., 2020).”
Simple and Efficient ways to Improve REALM (2021, methodology): “retrieval like BM25 or TF-IDF for retrieving passages/documents (Chen et al., 2017; Yang et al., 2019; Nie et al., 2019; Min et al., 2019b).”