Commonsense for Generative Multi-Hop Question Answering Tasks

By Lisa Bauer, Yicheng Wang, Mohit Bansal. 2018

191 citations recorded by the source corpus.

Reading comprehension QA tasks have seen a recent surge in popularity, yet most works have focused on fact-finding extractive QA. We instead focus on a more challenging multi-hop generative task (NarrativeQA), which requires the model to reason, gather, and synthesize disjoint pieces of information within the context to generate an answer. This type of multi-step reasoning also often requires understanding implicit relations, which humans resolve via external, background commonsense knowledge. We first present a strong generative baseline that uses a multi-attention mechanism to perform multiple hops of reasoning and a pointer-generator decoder to synthesize the answer. This model performs substantially better than previous generative models, and is competitive with current state-of-the-art span prediction models. We next introduce a novel system for selecting grounded multi-hop relational commonsense information from ConceptNet via a pointwise mutual information and term-frequency based scoring function. Finally, we effectively use this extracted commonsense information to fill in gaps of reasoning between context hops, using a selectively-gated attention mechanism. This boosts the model’s performance significantly (also verified via human evaluation), establishing a new state-of-the-art for the task. We also show that our background knowledge enhancements are generalizable and improve performance on QAngaroo-WikiHop, another multi-hop reasoning dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Knowledge Enhanced Contextual Word Representations (2019, background): “Task-specific KB architectures Other work has focused on integrating KBs into neural architectures for specific downstream tasks (Yang and Mitchell, 2017; Sun et al., 2018; Chen et al., 2018; Bauer et al., 2018; Mihaylov and Frank, 2018; Wang and Jiang, 2019; Yang et al., 2019).”
Unsupervised Commonsense Question Answering with Self-Talk (2020, background): “subgraphs from ConceptNet), and attending to it via attention mechanism when representing the inputs (Bauer et al., 2018; Paul and Frank, 2019; Lin et al., 2019).”
“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding (2019, background): “Natural language understanding requires the ability to reason with commonsense knowledge (Schu-bert, 2002; Davis, 2014), and the last few years have seen signiﬁcant amount of work in this direction (e.g., Zhang et al. (2017); Bauer et al. (2018); Tandon et al. (2018)).”
GraDA: Graph Generative Data Augmentation for Commonsense Reasoning (2022, methodology): “Data Preparation: To prepare graph-to-text datasets for training GraphGPT2, we map the questions to multi-hop paths in ConceptNet (Bauer et al., 2018).”
Question Answering by Reasoning Across Documents with Graph Convolutional Networks (2018, methodology): “0 – – MHPGM (Bauer et al., 2018) – 58.”
Making Language Models Better Reasoners with Step-Aware Verifier (2022, background): “Much research in the literature enhances the reasoning capabilities of machine learning systems by exploiting symbolic systems, including knowledge graphs (Mihaylov and Frank, 2018; Bauer et al., 2018; Kundu et al., 2019; Wang et al., 2019; Lin et al., 2019; Ding et al., 2019; Feng et al., 2020; Wan”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “[160] proposed multi-hop bidirectional attention and pointer-generator decoder for effective multihop reasoning and coherent answer generation, where external commonsense knowledge is utilized by relational path selection from ConceptNet and injection with selectivelygated attention.”
A Survey on Application of Knowledge Graph (2020, background): “Unlike other work which focuses on fact-finding extractive QA, the research work [21] focuses on multi-hop generative task.”
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, background): “Recently, multiple datasets have been proposed for multi-hop QA, in which questions can only be answered when considering information from multiple sentences and/or documents (Clark et al., 2018; Khashabi et al., 2018a; Yang et al., 2018; Welbl et al., 2018; Mihaylov et al., 2018; Bauer et al., 2018”
Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering (2019, methodology): “Inspired by Bauer et al. (2018), we propose to use attention-based injection.”
Temporal Common Sense Acquisition with Minimal Supervision (2020, background): “rarely say the obvious, in order to communicate more efficiently, but sometimes highlight rarities (Schubert, 2002; Van Durme, 2009; Gordon and Van Durme, 2013; Zhang et al., 2017; Bauer et al., 2018; Tandon et al., 2018).”
Multi-style Generative Reading Comprehension (2019, unknown): “, 2018) and generative (Bauer et al., 2018; Indurthi et al., 2018) models.”
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021, background): “Other works on scoring or pruning KG nodes/paths rely on graph-based metrics such as PageRank, centrality, and off-the-shelf KG embeddings (Paul and Frank, 2019; Fadnis et al., 2019; Bauer et al., 2018; Lin et al., 2019), without reflecting the QA context.”
Using Natural Language Relations between Answer Choices for Machine Comprehension (2019, background): “, 2018) or an external commonsense knowledge base (Das et al., 2017; Mihaylov and Frank, 2018; Bauer et al., 2018; Weissenborn et al., 2017).”
pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference (2018, background): “For cross-sentence inference tasks, Weissenborn et al. (2017), Bauer et al. (2018), and Mihaylov and Frank (2018) dynamically reﬁne word representations by reading assertions from ConceptNet and Wikipedia abstracts.”
Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations (2021, background): “Moreover, while knowledge grounding has been done for black-box models (Bauer et al., 2018; Chandu et al., 2021; Chen et al., 2020a), we are not aware of any work on knowledge grounding for self-rationalizing models.”
Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (2020, background): “…knowledge, either in the form of nodes (Yang and Mitchell, 2017; Wang et al., 2019), triples (Weissenborn et al., 2017; Mihaylov and Frank, 2018), paths (Bauer et al., 2018; Kundu et al., 2019; Lin et al., 2019), or sub-graphs (Li and Clark, 2015), and encode them to augment textual understanding.”
On the Advance of Making Language Models Better Reasoners (2022, background): “…the reasoning capabilities of machine learning systems by exploiting symbolic systems, including knowledge graphs (Mihaylov and Frank, 2018; Bauer et al., 2018; Kundu et al., 2019; Wang et al., 2019; Lin et al., 2019; Ding et al., 2019; Feng et al., 2020; Wang et al., 2022b), or question…”