QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
By Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, Jure Leskovec. 2021
863 citations recorded by the source corpus.
The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify relevant knowledge from large KGs, and (ii) perform joint reasoning over the QA context and KG. Here we propose a new model, QA-GNN, which addresses the above challenges through two key innovations: (i) relevance scoring, where we use LMs to estimate the importance of KG nodes relative to the given QA context, and (ii) joint reasoning, where we connect the QA context and KG to form a joint graph, and mutually update their representations through graph-based message passing. We evaluate QA-GNN on the CommonsenseQA and OpenBookQA datasets, and show its improvement over existing LM and LM+KG models, as well as its capability to perform interpretable and structured reasoning, e.g., correctly handling negation in questions.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Question Answering & Reading Comprehension, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Percy Liang
- Mohit Bansal
- Yejin Choi
- Graham Neubig
- Noah A. Smith
- Hannaneh Hajishirzi
- Caiming Xiong
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Luc Van Gool
- Mohit Bansal
- Trevor Darrell
- Xiaogang Wang
- Andrea Vedaldi
- Yejin Choi
- Jitendra Malik
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Denny Zhou
- Yejin Choi
- Hannaneh Hajishirzi
- Sean Welleck
- Xinyun Chen
- Michael Lewis
- Noah A. Smith
In Question Answering & Reading Comprehension
- Percy Liang
- Wen-tau Yih
- Hannaneh Hajishirzi
- Christopher D. Manning
- Yejin Choi
- Jure Leskovec
- Deng Cai
- Michihiro Yasunaga
In Knowledge Graphs & Representation
- Ronan Le Bras
- Jure Leskovec
- Yejin Choi
- Michihiro Yasunaga
- Sean Welleck
- Peter West
- Hongyu Ren
- Percy Liang
Citations in the corpus
- UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, methodology): “Structured knowledge encoding methods have been explored in many prior studies (Bogin et al., 2019; Lin et al., 2019; Yin et al., 2020a; Herzig et al., 2020; Agarwal et al., 2020; Saxena et al., 2020; Yasunaga and Liang, 2020; Yasunaga et al., 2021; Oguz et al., 2021).”
- Relational Memory-Augmented Language Models (2022, background): “Relation triples form the basis of knowledge bases, empowering a wide range of applications such as question answering (Yasunaga et al., 2021), machine reading (Yang and Mitchell, 2019), and reasoning (Minervini et al.”
- Transform-Retrieve-Generate: Natural Language-Centric Outside-Knowledge Visual Question Answering (2022, background): “There are mainly two streams of approaches, namely knowledge graph-based question answering [10, 32, 38, 52, 59, 65] and knowledge retrieval-based question answering [5, 7, 19, 29, 44, 45, 60, 62].”
- Exploiting Reasoning Chains for Multi-hop Science Question Answering (2021, methodology): “…explore the effectiveness of KG by either automatically constructing the graph using named entity or semantic role labeling (Qiu et al., 2019; Bosselut et al., 2019; Fang et al., 2020; Chen et al., 2019) or resorting to existing KG (Saxena et al., 2020; Zhang et al., 2020; Yasunaga et al., 2021).”
- Large Language Models as Analogical Reasoners (2023, background): “…et al., 2022b; Lightman et al., 2023; Moor et al., 2023), retrieving structured knowledge (Lin et al., 2019; Feng et al., 2020; Zhang et al., 2022a; Yasunaga et al., 2021; 2022a; Xie et al., 2022), and incorporating external modules for reasoning such as logic and program execution (Chen et al.,…”
- Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, methodology): “QA-GNN [131] uses LLMs to encode the question and candidate answer pairs, which are adopted to estimate the importance of relative KG entities.”
- Editing Large Language Models: Problems, Methods, and Opportunities (2023, neutral): “Moreover, some model editing techniques bear resemblance to knowledge augmentation (Zhang et al., 2019; Lewis et al., 2020; Zhang et al., 2022b; Yasunaga et al., 2021; Yao et al., 2022; Pan et al., 2023) approaches, as updating the model’s knowledge can also be considered as instilling knowledge…”
- Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (2023, methodology): “(Miller et al., 2016; He et al., 2021; Yasunaga et al., 2021).”
- Biases in Large Language Models: Origins, Inventory, and Discussion (2023, background): “The ield has witnessed a drastic change in paradigm with the advent and wide availability of large-scale pretrained language models, such as BERT [46], GPT [21, 108], T5 [109] and BART [79], which are now pervasive in every high-performance system for Machine Translation [25], Question Answering [95”
- A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, methodology): “For example, QA-GNN [264] first attempts to use LLMs for text preprocessing and further guides the reasoning step on the KGs.”
- Generated Knowledge Prompting for Commonsense Reasoning (2021, background): “Another direction is to ground the question into a knowledge graph and do inference with graph-based reasoning (Lin et al., 2019; Lv et al., 2020; Yasunaga et al., 2021).”
- Retrieval-Augmented Multimodal Language Modeling (2023, background): “, 2022) or a structured knowledge base (Zhang et al., 2019; Agarwal et al., 2021; Xie et al., 2022; Yasunaga et al., 2021; 2022a).”
- Knowledge Graph Prompting for Multi-Document Question Answering (2023, background): “• Existing Knowledge Base : One common approach in the literature is to use existing knowledge bases or extract subgraphs from them for specific tasks (Yasunaga et al. 2022; Dong et al. 2023; Yasunaga et al. 2021).”
- GreaseLM: Graph REASoning Enhanced Language Models for Question Answering (2022, methodology): “…73.5 ( ± 0.2) 69.0 ( ± 0.8) RN (Santoro et al., 2017) 74.6 ( ± 0.9) 69.1 ( ± 0.2) MHGRN (Feng et al., 2020) 74.5 ( ± 0.1) 71.1 ( ± 0.8) QA-GNN (Yasunaga et al., 2021) 76.5 ( ± 0.2) 73.4 ( ± 0.9) G REASE LM ( Ours ) 78.5 ( ± 0.5) 74.2 ( ± 0.4) et al., 2020) 81.4 ≥ 355M AMR-SG (Xu et al., 2021)…”
- Deep Bidirectional Language-Knowledge Graph Pretraining (2022, positive): “Across all tasks, D RAGON consistently outperforms the existing LM (RoBERTa) and KG-augmented QA models (QAGNN, GreaseLM), e.g., +7% absolute accuracy boost over RoBERTa and +5% over GreaseLM on OBQA .”