Reasoning with Memory Augmented Neural Networks for Language Comprehension

By Tsendsuren Munkhdalai, Hong Yu. 2016

26 citations recorded by the source corpus.

Hypothesis testing is an important cognitive process that supports human reasoning. In this paper, we introduce a computational hypothesis testing approach based on memory augmented neural networks. Our approach involves a hypothesis testing loop that reconsiders and progressively refines a previously formed hypothesis in order to generate new hypotheses to test. We apply the proposed approach to language comprehension task by using Neural Semantic Encoders (NSE). Our NSE models achieve the state-of-the-art results showing an absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by single and ensemble systems on standard machine comprehension benchmarks such as the Children's Book Test (CBT) and Who-Did-What (WDW) news article datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Recurrent Neural Networks for Classifying Relations in Clinical Notes (2017, neutral): “In general domain NLP, a growing number of studies have successfully used recurrent neural networks (RNNs) combined with word embedding [22] on tasks including language modeling [23], text classification [24–27], question answering [25,26,28,29], machine translation [25,30–32], named entity recognit”
Tracking the World State with Recurrent Entity Networks (2016, methodology): “The CommNN model of (Sukhbaatar et al., 2016) also uses a set of parallel recurre nt models with tied weights, but differs from our model in their use of inter-network communication a nd the lack of a gating mechanism.”
Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge (2018, neutral): “In contrast to many previous complex models (Weston et al., 2015; Dhingra et al., 2017; Cui et al., 2017; Munkhdalai and Yu, 2016; Sordoni et al., 2016) that perform multi-turn reading of a story and a question before inferring the correct answer, we aim to tackle the cloze-style RC task in a way th”
Linguistic Knowledge as Memory for Recurrent Neural Networks (2017, unknown): “Several large-scale benchmarks (Rajpurkar et al., 2016; Onishi et al., 2016) and deep learning models (Munkhdalai & Yu, 2016) have been proposed for this task recently.”
Subword-augmented Embedding for Cloze Reading Comprehension (2018, background): “Recently, many deep learning models have been proposed for reading comprehension (Sordoni et al., 2016; Trischler et al., 2016; Wang and Jiang, 2016; Munkhdalai and Yu, 2017; Wang et al., 2017a; Dhingra et al., 2017; Zhang et al., 2018b; Wang et al., 2018b).”
Effective Character-Augmented Word Embedding for Machine Reading Comprehension (2018, background): “Machine reading comprehension has been witnessed rapid progress in recent years [26,27,29,22,28,8,33,35,31].”
Dynamically Fused Graph Network for Multi-hop Reasoning (2019, background): “Munkhdalai and Yu (2017) and Onishi et al. (2016) incorporate a hypothesis testing loop to update the query representation at each reasoning step and select the best answer among the candidate entities at the last step.”