PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text
By Haitian Sun, Tania Bedrax-Weiss, William W. Cohen. 2019
464 citations recorded by the source corpus.
We consider open-domain question answering (QA) where answers are drawn from either a corpus, a knowledge base (KB), or a combination of both of these. We focus on a setting in which a corpus is supplemented with a large but incomplete KB, and on questions that require non-trivial (e.g., “multi-hop”) reasoning. We describe PullNet, an integrated framework for (1) learning what to retrieve and (2) reasoning with this heterogeneous information to find the best answer. PullNet uses an iterative process to construct a question-specific subgraph that contains information relevant to the question. In each iteration, a graph convolutional network (graph CNN) is used to identify subgraph nodes that should be expanded using retrieval (or “pull”) operations on the corpus and/or KB. After the subgraph is complete, another graph CNN is used to extract the answer from the subgraph. This retrieve-and-reason process allows us to answer multi-hop questions using large KBs and corpora. PullNet is weakly supervised, requiring question-answer pairs but not gold inference paths. Experimentally PullNet improves over the prior state-of-the art, and in the setting where a corpus is used with incomplete KB these improvements are often dramatic. PullNet is also often superior to prior systems in a KB-only setting or a text-only setting.
Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Percy Liang
- Jianfeng Gao
- Hannaneh Hajishirzi
- Caiming Xiong
- Yejin Choi
- Graham Neubig
In Retrieval-Augmented Generation
- Hannaneh Hajishirzi
- Danqi Chen
- Akari Asai
- Wen-tau Yih
- Jason Aaron Edward Weston
- William W. Cohen
- Sewon Min
- Bhuwan Dhingra
In Natural Language Processing
- Mohit Bansal
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Percy Liang
- Caiming Xiong
- Graham Neubig
- Yejin Choi
- Jianfeng Gao
Citations in the corpus
- Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (2019, background): “PullNet (Sun et al., 2019) learns to retrieve question-aware sub-graphs from text corpora and knowledge bases (e.g., Freebase), while we focus on open-domain QA solely based on text.”
- Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, background): “However, recent work has augmented knowledge bases with text from Wikipedia (Das et al., 2017; Sun et al., 2018, 2019), to increase factual coverage.”
- Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “These recent neural approaches over unstructured text have overtaken prior methods exploiting the graph structure of knowledge sources (such as hyperlinks in Wikipedia) (Min et al., 2019; Asai et al., 2020; Sun et al., 2019; Xiong et al., 2019), and are an attractive alternative for dialogue.”
- Differentiable Reasoning over a Virtual Knowledge Base (2020, methodology): “Even with only 50% edges retained, we see good performance—better than PullNet and the state-of-the-art for a KB-only method (in italics ).”
- Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge (2020, background): “…has also been extended with works looking at ways of incorporating raw text and symbolic KGs into a shared embedding space (Riedel et al., 2013; Verga et al., 2016), to be jointly reasoned over (Sun et al., 2018, 2019a), or to treat text as a replacement for a knowledge base (Dhingra et al., 2019).”
- HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data (2020, background): “PullNet (Sun et al., 2019) improves over GRAFT to use an integrated framework that dynamically learns to retrieve and reason over heterogeneous information to find the best answers.”
- UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering (2020, methodology): “The latest in this line are the works of (Sun et al., 2018, 2019), which augments the knowledge graph with text nodes and applies graph methods to identify candidate answers.”
- Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings (2020, methodology): “Recently, there has been some work in which text corpus is incorporated as a knowledge source in addition to KG to answer complex questions on KGs (Sun et al., 2018, 2019a).”
- Neural, symbolic and neural-symbolic reasoning on knowledge graphs (2021, methodology): “Although some models such as Graft-Net (Sun et al., 2018) and PullNet (Sun et al., 2019a) are the state-of-the-art reasoning models on both the structural and the textual information, it is still challenging to determine the correct evidence to be linked to the incomplete graph structure from the la”
- LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs (2021, methodology): “Compared with non-parsing methods (Sun et al., 2019a; Saxena et al., 2020), these models achieve better empirical results and interpretability since the underlying structured query captures the reasoning process.”
- Open-Domain Question Answering (2020, background): “…data to guide the retriever or reader stage of existing textual QA systems (Asai et al., 2020; Min et al., 2019b), or (2) how to synthesize information from these two heterogeneous sources and build effective QA models on the combined information (Sun et al., 2018, 2019; Xiong et al., 2019).”
- Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering (2021, background): “In recent years, with the popularity of Graph Neural Network (GNN), some works [87], [119], [120] propose to gain relevant information not only from a text corpus but also from a KB to facilitate evidence retrieval and question answering.”
- Differentiable Open-Ended Commonsense Reasoning (2020, background): “Multi-Hop Reasoning Many recent models for open-domain QA tackle multi-hop reasoning through iterative retrieval, e.g., GRAFT-Net (Sun et al., 2018), MUPPET (Feldman and El-Yaniv, 2019), PullNet (Sun et al., 2019), and GoldEn (Qi et al., 2019).”
- Deep Bidirectional Language-Knowledge Graph Pretraining (2022, unknown): “…predicts for the text side, link prediction holds out some edges and predicts them for the input KG. Link prediction is a fundamental task in KGs (Sun et al. 2019b) and makes the model use the structure of KGs to perform reasoning (e.g., using a compositional path “X’s mother’s husband is Y” to…”
- Multi-Modal Answer Validation for Knowledge-Based VQA (2021, methodology): “We used the constituent parser from AllenNLP to extract the nouns phrases in the question.”
- Question Answering Over Temporal Knowledge Graphs (2021, methodology): “To the best of our knowledge, recent KGQA algorithms (Miller et al. 2016; Sun et al. 2019; Cohen et al. 2020; Sun et al. 2020) work with non-”
- Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, background): “…: KB completion is an extensively studied research field (Nickel et al., 2011; Bordes et al., 2013; Socher et al., 2013; Velickovic et al., 2018; Sun et al., 2019b) and several methods have been developed that learn low dimensional representation of relations such that similar relations are…”