Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering

By Peifeng Wang, Nanyun Peng, Pedro A. Szekely, Xiang Ren. 2020

94 citations recorded by the source corpus.

Commonsense question answering (QA) requires background knowledge which is not explicitly stated in a given context. Prior works use commonsense knowledge graphs (KGs) to obtain this knowledge for reasoning. However, relying entirely on these KGs may not suffice, considering their limited coverage and the contextual dependence of their knowledge. In this paper, we augment a general commonsense QA framework with a knowledgeable path generator. By extrapolating over existing paths in a KG with a state-of-the-art language model, our generator learns to connect a pair of entities in text with a dynamic, and potentially novel, multi-hop relational path. Such paths can provide structured evidence for solving commonsense questions without fine-tuning the path generator. Experiments on two datasets show the superiority of our method over previous works which fully rely on knowledge from KGs (with up to 6% improvement in accuracy), across various amounts of training data. Further evaluation suggests that the generated paths are typically interpretable, novel, and relevant to the task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention (2021, methodology): “Previous methods for natural language understanding (NLU) (Xu et al., 2021; Yan et al., 2020; Wang et al., 2020; Khashabi et al., 2020) have tried using BERT (Devlin et al.”
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021, background): “Several works use GNNs to model the structure of text (Yasunaga et al., 2017; Zhang et al., 2018; Yasunaga and Liang, 2020) or KGs (Wang et al., 2020a).”
GreaseLM: Graph REASoning Enhanced Language Models for Question Answering (2022, methodology): “Certain approaches learn to access implicit knowledge encoded in LMs (Bosselut et al., 2019; Petroni et al., 2019; Hwang et al., 2021) by training on structured KG data, and then use the LM to generate local KGs that can be used for QA (Wang et al., 2020; Bosselut et al., 2021).”
Kformer: Knowledge Injection in Transformer Feed-Forward Layers (2022, methodology): “To obtain the knowledge representation, [18] computes the knowledge embedding using a Relational Network to aggregate the retrieved paths while [1] encodes the knowledge by retrofitting the pre-trained word embedding.”