KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning
By Bill Yuchen Lin, Xinyue Chen, Jamin Chen, Xiang Ren. 2019
528 citations recorded by the source corpus.
Commonsense reasoning aims to empower machines with the human ability to make presumptions about ordinary situations in our daily life. In this paper, we propose a textual inference framework for answering commonsense questions, which effectively utilizes external, structured commonsense knowledge graphs to perform explainable inferences. The framework first grounds a question-answer pair from the semantic space to the knowledge-based symbolic space as a schema graph, a related sub-graph of external knowledge graphs. It represents schema graphs with a novel knowledge-aware graph network module named KagNet, and finally scores answers with graph representations. Our model is based on graph convolutional networks and LSTMs, with a hierarchical path-based attention mechanism. The intermediate attention scores make it transparent and interpretable, which thus produce trustworthy inferences. Using ConceptNet as the only external resource for Bert-based models, we achieved state-of-the-art performance on the CommonsenseQA, a large-scale dataset for commonsense reasoning.
Subfields: Artificial Intelligence, Graph Learning, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Percy Liang
- Luke S. Zettlemoyer
- Jianfeng Gao
- Hannaneh Hajishirzi
- Caiming Xiong
- Noah A. Smith
In Graph Learning
- Shirui Pan
- Philip S. Yu
- Percy Liang
- Jure Leskovec
- Xiang Ren
- Michihiro Yasunaga
- Guodong Long
- William L. Hamilton
In Knowledge Graphs & Representation
- Xiang Ren
- Jure Leskovec
- Michihiro Yasunaga
- Ronan Le Bras
- Percy Liang
- Hongyu Ren
- Yejin Choi
- Antoine Bosselut
Citations in the corpus
- UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, methodology): “Structured knowledge encoding has been widely explored (Bogin et al., 2019; Lin et al., 2019; Agar-wal et al., 2020; Saxena et al., 2020; Yasunaga and Liang, 2020; Yasunaga et al., 2022; and others detailed in Section 2).”
- CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020, background): “We argue that imposing commonsense knowledge with additional graph structures (Lin et al., 2019) between input concepts is a more promising future direction for the COMMONGEN task as graphs are naturally order-insensitive.”
- Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021, background): “Numerous models have included knowledge resources such as ConceptNet (Speer et al., 2017) for question answering (Lin et al., 2019), sarcasm generation (Chakrabarty et al., 2020), and dialogue response generation (Zhou et al., 2018, 2021), among others.”
- Unsupervised Commonsense Question Answering with Self-Talk (2020, background): “subgraphs from ConceptNet), and attending to it via attention mechanism when representing the inputs (Bauer et al., 2018; Paul and Frank, 2019; Lin et al., 2019).”
- Birds Have Four Legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models (2020, methodology): “Lin et al. (2019a) also show that PTLMs can retrieve paths from ConceptNet that aid in interpreting the decision made by the PTLMs on the CommonsenseQA dataset (Talmor et al., 2019b).”
- Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering (2020, methodology): “They further score or rank the retrieved paths using graph-based metrics (e,g., PageRank, centrality) (Paul and Frank, 2019; Fad-nis et al., 2019; Bauer et al., 2018), handcrafted rules (Kapanipathi et al., 2019) or neural methods (e.g., attention mechanisms) (Kundu et al., 2018; Lin et al., 2019).”
- Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (2020, methodology): “To further equip RN with the ability to model nondegenerate paths, KagNet (Lin et al., 2019) adopts LSTMs to encode all paths connecting question entities and answer entities with lengths no more than K.”
- TransOMCS: From Linguistic Graphs to Commonsense Knowledge (2020, background): “As an important knowledge resource for AI systems, commonsense knowledge has been shown helpful in many downstream tasks such as question answering [Lin et al., 2019] and reading comprehension [Wang et al.”
- COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs (2020, background): “2 Background Commonsense Knowledge Graphs Large scale commonsense knowledge graphs are ubiquitous tools in natural language processing tasks as access to their facts allows models to learn to reason over commonsense knowledge to make predictions (Lin et al. 2019; Feng et al. 2020).”
- Generated Knowledge Prompting for Commonsense Reasoning (2021, background): “Another direction is to ground the question into a knowledge graph and do inference with graph-based reasoning (Lin et al., 2019; Lv et al., 2020; Yasunaga et al., 2021).”
- Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, methodology): “15, KagNet [38] and QA-GNN [118] make the results generated by LLMs at each reasoning step grounded by knowledge graphs.”
- Think about it! Improving defeasible reasoning by first modeling the question scenario. (2021, methodology): “Encoding Graph Representations Several existing methods use graphs as an additional input for commonsense reasoning (Sun et al., 2018; Lin et al., 2019; Lv et al., 2020; Feng et al., 2020; Bosselut et al., 2021; Ma et al., 2021; Kapani-pathi et al., 2020).”
- Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning (2020, methodology): “Our approach is not directly comparable to Lv et al. (2019) and Lin et al. (2019), the first of which achieves 75.3% accuracy.”
- WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference (2020, methodology): “Approaches to either use pre-existing frame resources such as FrameNet (Baker et al., 1998) for multi-hop inference (D’Souza et al., 2019), or to extract small subsets of knowledge graphs to serve as proxies for schemata (Lin et al., 2019), have shown initial progress for multi-hop reasoning.”
- GreaseLM: Graph REASoning Enhanced Language Models for Question Answering (2022, methodology): “…and context from language to affect fine-grained entity knowledge representations in the GNN. methods propose various ways to leverage both modalities ( i.e. , expressive large language models and structured KGs) for improved reasoning (Mihaylov & Frank, 2018; Lin et al., 2019; Feng et al., 2020).”
- Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer (2022, background): “…external knowledge bases (e.g., knowledge graph, dictionary) to inject entity knowledge into PTLMs (Zhang et al., 2019; Peters et al., 2019; Wang et al., 2021; Yu et al., 2021) or retrieve knowledge from external knowledge bases to solve the problem (Lin et al., 2019; Wang et al., 2020).”
- Making Language Models Better Reasoners with Step-Aware Verifier (2022, background): “…by exploiting symbolic systems, including knowledge graphs (Mihaylov and Frank, 2018; Bauer et al., 2018; Kundu et al., 2019; Wang et al., 2019; Lin et al., 2019; Ding et al., 2019; Feng et al., 2020; Wang et al., 2022b), or question taxonomies (Dua et al., 2019; Andor et al., 2019; Hu et al.,…”
- Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering (2019, methodology): “More recently, Lin et al. (2019) proposed to extract sub-graphs from ConceptNet and embed the knowledge using Graph Convolutional Networks (Kipf and Welling, 2016).”