Neural Models for Reasoning over Multiple Mentions Using Coreference

By Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov. 2018

108 citations recorded by the source corpus.

Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text. Existing Recurrent Neural Network (RNN) layers are biased towards short-term dependencies and hence not suited to such tasks. We present a recurrent layer which is instead biased towards coreferent dependencies. The layer uses coreference annotations extracted from an external system to connect entity mentions belonging to the same cluster. Incorporating this layer into a state-of-the-art reading comprehension model improves performance on three datasets – Wikihop, LAMBADA and the bAbi AI tasks – with large gains when training data is scarce.

Subfields: Artificial Intelligence, Question Answering & Reading Comprehension, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Question Answering & Reading Comprehension

In Information Extraction

Citations in the corpus

Multi-hop Reading Comprehension through Question Decomposition and Rescoring (2019, methodology): “Prior methods on multi-hop reading comprehension focus on answering relational queries, and emphasize attention models that reason over coreference chains (Dhingra et al., 2018; Zhong et al., 2019; Cao et al., 2019).”
Differentiable Reasoning over a Virtual Knowledge Base (2020, background): “Recent papers have also looked at complex QA using graph neural networks (Sun et al., 2018; Cao et al., 2019; Xiao et al., 2019) or by identifying paths of entities in text (Jiang et al., 2019; Kundu et al., 2019; Dhingra et al., 2018).”
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language (2019, methodology): “…literature, such multi-hop reading comprehension tasks are frequently solved via end-to-end differentiable (deep learning) models (Sukhbaatar et al., 2015; Peng et al., 2015; Seo et al., 2016b; Raison et al., 2018; Henaff et al., 2016; Kumar et al., 2016; Graves et al., 2016; Dhingra et al., 2018).”
Hierarchical Graph Network for Multi-hop Question Answering (2019, methodology): “For example, MHQA-GRN (Song et al., 2018) and Coref-GRN (Dhingra et al., 2018) construct an entity graph based on co-reference resolution or sliding windows.”
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, background): “The main work we are aware of is (Dhingra et al. 2018), which uses coreference clusters to improve reading comprehension and the works on language modeling by Ji et al.”
Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents (2019, methodology): “Multi-hop multi-document QA: The work in (Dhingra et al. 2018) designed a recurrent layer to explicitly exploit the skip connections between entities from different documents given coreference predictions.”
Commonsense for Generative Multi-Hop Question Answering Tasks (2018, methodology): “Similar to previous works (Dhingra et al., 2018), we use the non-oracle, unmasked and not-validated dataset.”
Understanding Dataset Design Choices for Multi-hop Reasoning (2019, result): “Table 2: The results of our no-context baseline compared with Coref-GRU (Dhingra et al., 2018), MHQAGRN (Song et al.”
Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures (2020, background): “Similarly, Cao et al. (2019) and Dhingra et al. (2018) incorporate linguistic information, i.e., coreference relations, in their model and show improvements in in-domain evaluations.”
Is Graph Structure Necessary for Multi-hop Reasoning? (2020, unknown): “Dhingra et al. (2018) and Song et al. (2018) designed a DAG-styled recurrent layer to model the relations between entities.”
Dynamically Fused Graph Network for Multi-hop Reasoning (2019, background): “, 2018b), have been previously studied and show promising results in QA tasks requiring reasoning (Dhingra et al., 2018; De Cao et al., 2018; Song et al., 2018a).”
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering (2019, background): “While there are several works that discuss solutions for multi-hop reasoning (Dhingra et al., 2018; Zhong et al., 2019), to the best of our knowledge, this work is the ﬁrst to propose a viable so-lution for open-domain multi-hop QA.”
The Referential Reader: A Recurrent Entity Network for Anaphora Resolution (2019, background): “The incorporation of entities into language models has also been explored in prior work (Yang et al., 2017; Kobayashi et al., 2017); similarly, Dhingra et al. (2018) augment the gated recurrent unit (GRU) architecture with additional edges be-tween coreferent mentions.”
Exploring Graph-structured Passage Representation for Multi-hop Reading Comprehension with Graph Neural Networks (2018, methodology): “This can be considered as a reimplementation of Dhingra et al. (2018) based on our framework.”
Evidence Sentence Extraction for Machine Reading Comprehension (2019, methodology): “Besides widely used lexical features such as part-of-speech tags and named entity types (Wang et al., 2016; Liu et al., 2017; Dhingra et al., 2017b, 2018), we consider more diverse types of external knowledge for performance improvements.”
Question Answering by Reasoning Across Documents with Graph Convolutional Networks (2018, methodology): “Note that Welbl et al. (2018) (Welbl et al., 2018) 74.1 – – – FastQA (Welbl et al., 2018) 25.7 – 35.8 – BiDAF (Welbl et al., 2018) 42.9 – 54.5 – Coref-GRU (Dhingra et al., 2018) 59.3 56.0 – – MHPGM (Bauer et al., 2018) – 58.2 – – Weaver / Jenga (Raison et al., 2018) 65.3 64.1 – – MHQA-GRN (Song et…”
Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension (2019, background): “Recently, Dhingra et al. (2018) leveraged coreference annotations from an external system to connect the entities.”