KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering

By Donghan Yu, Chenguang Zhu, Yuwei Fang, W. Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng. 2021

112 citations recorded by the source corpus.

Current Open-Domain Question Answering (ODQA) models typically include a retrieving module and a reading module, where the retriever selects potentially relevant passages from open-source documents for a given question, and the reader produces an answer based on the retrieved passages. The recently proposed Fusion-in-Decoder (FiD) framework is a representative example, which is built on top of a dense passage retriever and a generative reader, achieving the state-of-the-art performance. In this paper we further improve the FiD approach by introducing a knowledge-enhanced version, namely KG-FiD. Our new model uses a knowledge graph to establish the structural relationship among the retrieved passages, and a graph neural network (GNN) to re-rank the passages and select only a top few for further processing. Our experiments on common ODQA benchmark datasets (Natural Questions and TriviaQA) demonstrate that KG-FiD can achieve comparable or better performance in answer prediction than FiD, with less than 40% of the computation cost.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, unknown): “KG-FiD [306] conducts the retrieval and generation as normal FiD.”
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (2023, background): “Other lines of studies retrieving triples from KGs via BM25 (Li et al., 2023) or DPR (Karpukhin et al., 2020; Yu et al., 2022b) to improve the performance of LLMs.”
Knowledge-Augmented Methods for Natural Language Processing (2023, methodology): “Yu et al. (2022c) and Xu et al. (2021) ∗ propose to use dictionary descriptions as additional knowledge source for natural language understanding and commonsense reasoning tasks.”
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference (2022, background): “KG-FiD (Yu et al., 2022a) reranks retrieved passages and reads only the top passages, while Varshney et al. (2022) reads more retrieved passages only if it is not confident in its answer.”
Generate rather than Retrieve: Large Language Models are Strong Context Generators (2022, background): “, 2022) or the reader (Izacard & Grave, 2021; Cheng et al., 2021; Yu et al., 2022), or training the system end-to-end (Lewis et al.”
A Unified Encoder-Decoder Framework with Entity Memory (2022, methodology): “A popular approach to incorporate knowledge into the generation process is retrieving evidence documents from external sources (Lewis et al., 2020b; Izacard and 2021; Oguz et al., 2020; Yu et al., 2022c).”
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022, methodology): “Incorporating external knowledge is essential for many NLG tasks to augment the limited textual information (Yu et al., 2022c; Dong et al., 2021; Yu et al., 2022b).”
A Survey of Knowledge-enhanced Text Generation (2020, result): “, the decoder performs attention over the concatenation of the resulting representations of all retrieved passages [68, 140]) could even outperform RAG as reported in KILT benchmark [96].”
LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs (2024, methodology): “…et al., 2020a; Qu et al., 2020; Xiong et al., 2020b; Khalifa et al., 2023), enhancing the reader (Izacard and Grave, 2020b; Cheng et al., 2021; Yu et al., 2021; Borgeaud et al., 2022), fine-tuning the re-triever and reader jointly (Yu, 2022; Izacard et al., 2022; Singh et al., 2021; Izacard…”