Recurrent Memory Network for Language Modeling

By Ke M. Tran, Arianna Bisazza, Christof Monz. 2016

21 citations recorded by the source corpus.

Recurrent Neural Networks (RNN) have obtained excellent result in many natural language processing (NLP) tasks. However, understanding and interpreting the source of this success remains a challenge. In this paper, we propose Recurrent Memory Network (RMN), a novel RNN architecture, that not only amplifies the power of RNN but also facilitates our understanding of its internal functioning and allows us to discover underlying patterns in data. We demonstrate the power of RMN on language modeling and sentence completion tasks. On language modeling, RMN outperforms Long Short-Term Memory (LSTM) network on three large German, Italian, and English dataset. Additionally we perform in-depth analysis of various linguistic dimensions that RMN captures. On Sentence Completion Challenge, for which it is essential to capture sentence coherence, our RMN obtains 69.2% accuracy, surpassing the previous state-of-the-art by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Long Short-Term Memory-Networks for Machine Reading (2016, background): “Concurrent to our work is the pre-print Recurrent Memory Networks (Tran et al., 2016), which equips an LSTM with an external memory block interacting with the hidden state of the LSTM.”
Episodic CAMN: Contextual Attention-Based Memory Networks with Iterative Feedback for Scene Labeling (2017, background): “Neural Memory Networks [24, 54, 57, 60] became popular with the aid of attention mechanism where separate memory structures are involved to stably store information.”