Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge

By Todor Mihaylov, Anette Frank. 2018

184 citations recorded by the source corpus.

We introduce a neural reading comprehension model that integrates external commonsense knowledge, encoded as a key-value memory, in a cloze-style setting. Instead of relying only on document-to-question interaction or discrete features as in prior work, our model attends to relevant external knowledge and combines this knowledge with the context representation before inferring the answer. This allows the model to attract and imply knowledge from an external knowledge source that is not explicitly stated in the text, but that is relevant for inferring the answer. Our model improves results over a very strong baseline on a hard Common Nouns dataset, making it a strong competitor of much more complex models. By including knowledge explicitly, our model can also provide evidence about the background knowledge used in the RC process.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering (2018, methodology): “Building upon a recent neural model for incorporating external knowledge in the story cloze setting (Mihaylov and Frank, 2018), we propose a knowledge-aware neural baseline that can utilize both the open book F and common knowledge retrieved from sources such as ConceptNet (Speer et al.”
Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering (2019, methodology): “Previous work has converted KB facts to sentences to provide extra evidence (Weissenborn et al., 2017; Mihaylov and Frank, 2018), but do not explicitly use the KB graph structure.”
ERNIE: Enhanced Language Representation with Informative Entities (2019, background): “…it outputs new token embeddings and entity embeddings for the next layer. signiﬁcantly enhance original models, such as reading comprehension (Mihaylov and Frank, 2018; Zhong et al., 2018), machine translation (Zaremoodi et al., 2018), natural language inference (Chen et al., 2018),…”
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation (2019, background): “, 2018), reading comprehension (Mihaylov and Frank, 2018; Zhong et al., 2019) and dialogue system (Madotto et al.”
Unsupervised Question Answering by Cloze Translation (2019, background): “Finally, injecting external knowledge into QA systems could be viewed as semi-supervision, and Weissenborn et al. (2017) and Mihaylov and Frank (2018) use Conceptnet (Speer et al., 2016) for QA tasks.”
Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations (2019, background): “…Pineau, 2018; Liu et al., 2018; Moghe et al., 2018; Dinan et al., 2019; Zhong et al., 2019), task-oriented dialogue systems (Madotto et al., 2018; Wu et al., 2019; He et al., 2019) and question answering systems (Kiddon et al., 2016; Hao et al., 2017; Sun et al., 2018; Mihaylov and Frank, 2018).”
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021, background): “To provide more explicit and interpretable knowledge, several works integrate structured knowledge (KGs) into LMs (Mihaylov and Frank, 2018; Lin et al., 2019; Wang et al., 2019a; Yang et al., 2019; Wang et al., 2020b; Bosselut et al., 2021).”
Using Natural Language Relations between Answer Choices for Machine Comprehension (2019, background): “, 2018) or an external commonsense knowledge base (Das et al., 2017; Mihaylov and Frank, 2018; Bauer et al., 2018; Weissenborn et al., 2017).”
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation (2020, background): “…is necessary and beneﬁcial for language inference (LoBue and Yates, 2011; Bowman et al., 2015; Rashkin et al., 2018b), reading comprehension (Mihaylov and Frank, 2018; Rashkin et al., 2018a), and particularly for open-ended language generation, which usually requires external knowledge to…”
Incorporating External Knowledge into Machine Reading for Generative Question Answering (2019, background): “Mihaylov and Frank (2018) showed improvements on a cloze-style task by incorporating commonsense knowledge via a context-to-commonsense attention.”
Explicit Utilization of General Knowledge in Machine Reading Comprehension (2018, background): “The existing way to solve the second problem is to encode general knowledge in vector space so that the encoding results can be used to enhance the lexical or contextual representations of words (Weissenborn et al., 2017; Mihaylov and Frank, 2018).”
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering (2020, background): “Existing systems retrieve knowledge from a KG in the form of: triplets (Mihaylov and Frank, 2018), multihop paths (Lin et al.”
Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension (2019, background): “Weissenborn et al. (2017), Bauer et al. (2018), Mihaylov and Frank (2018), Pan et al. (2019), Chen et al. (2018), Wang et al. (2018) follow a retrieve-then-encode paradigm, i.e., they ﬁrst retrieve relevant knowledge from KBs, and only the retrieved knowledge relevant locally to the reading text…”
Natural Language Processing with Commonsense Knowledge: A Survey (2021, background): “Mihaylov and Frank [125] developed a cloze-style reading comprehension model that encodes relevant commonsense knowledge from ConceptNet as a key-value memory.”
Knowledge Aware Emotion Recognition in Textual Conversations via Multi-Task Incremental Transformer (2020, background): “…up around the theme of incorporating external knowledge in generative conversation systems, including question answering systems (Hao et al., 2017; Mihaylov and Frank, 2018), open-domain dialogue systems (Young et al., 2018; Zhou et al., 2018b; Zhong et al., 2019a), and task-oriented dialogue…”