Generative Temporal Models with Memory

By Mevlana Gemici, Chia-Chun Hung, Adam Santoro, Greg Wayne, Shakir Mohamed, Danilo Jimenez Rezende, David Amos, Timothy P. Lillicrap. 2017

57 citations recorded by the source corpus.

We consider the general problem of modeling temporal data with long-range dependencies, wherein new observations are fully or partially predictable based on temporally-distant, past observations. A sufficiently powerful temporal model should separate predictable elements of the sequence from unpredictable elements, express uncertainty about those unpredictable elements, and rapidly identify novel elements that may help to predict the future. To create such models, we introduce Generative Temporal Models augmented with external memory systems. They are developed within the variational inference framework, which provides both a practical training methodology and methods to gain insight into the models' operation. We show, on a range of problems with sparse, long-term temporal dependencies, that these models store information from early in a sequence, and reuse this stored information efficiently. This allows them to perform substantially better than existing models based on well-known recurrent neural networks, like LSTMs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Time-Series & Forecasting

Citations in the corpus

Learning Latent Dynamics for Planning from Pixels (2018, background): “In simulated environments, Gemici et al. (2017) augment dynamics models with an external memory to remember long-time contexts. van den Oord et al. (2017) propose a variational model that avoids sampling using a nearest neighbor look-up, yielding high ﬁdelity image predictions.”
Recurrent World Models Facilitate Policy Evolution (2018, methodology): “Future work will explore replacing the VAE and MDN-RNN with higher capacity models [27, 89, 93, 97, 98], or incorporating an external memory module [19, 107], if we want our agent to learn to explore more complicated worlds.”
World Models (2018, background): “, 2017), or incorporating an external memory module (Gemici et al., 2017), if we want our agent to learn to explore more complicated worlds.”
Variational Memory Addressing in Generative Models (2017, methodology): “Generative models with memory have also been proposed for sequence modelling in [9], using differentiable soft addressing.”
A General Method for Amortizing Variational Filtering (2018, methodology): “For instance, many filtering inference methods re-use various components of the generative model [7, 12, 15, 9], while some methods introduce separate recurrent neural networks into the filtering procedure [4, 9] or encode the previous latent sample [26].”
The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation (2019, methodology): “This allows us to use modern Bayesian methods (Kingma and Welling, 2013; Gemici et al., 2017) to learn the network weights and perform inference on g and p.”
Generating Sentences Using a Dynamic Canvas (2018, methodology): “To this end we introduce the A ttentive U nsupervised T ext (W) r iter (AUTR), which is a word level generative model for text; AUTR uses an RNN with a dynamic attention mechanism to iteratively update a canvas (analogous to an external memory (Gemici et al. 2017)).”
Neural Machine Translation (2020, unknown): “There is a line of research under the banner neural Turing machines that explores what kind of architectures are needed to implement basic algorithms (Gemici et al., 2017).”
Small Sample Learning in Big Data Era (2018, background): “, 2017c), and imagination of realistic environment (Chiappa et al., 2017; Racanière et al., 2017; Gemici et al., 2017; Hamrick et al., 2017) based on simulation-based planning and imagination, which leads to data efficiency improvement and novel concept learning.”
Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning (2017, background): “Proposed solutions are recurrent neural networks (RNN) or Neural Turing Machines (NTM), which were both studied with variational inference context as well (in [6] and [12], respectively).”
Probing Physics Knowledge Using Tools from Developmental Psychology (2018, methodology): “Unless where stated otherwise, implementation details are exactly as in the LRU model of Gemici et al. (2017).”
Probabilistic Recurrent State-Space Models (2018, unknown): “A variety of stochastic deep recurrent models has been presented based on Stochastic Gradient Variational Bayes (SGVB) (Bayer & Osendorfer, 2014; Krishnan et al., 2015; Watter et al., 2015; Chung et al., 2015; Archer et al., 2015; Karl et al., 2016; Fraccaro et al., 2016; Gemici et al., 2017).”
Mastering Atari with Discrete World Models (2020, methodology): “, 2019; Wang and Ba, 2019), model images without using them for planning (Oh et al., 2015; Krishnan et al., 2015; Karl et al., 2016; Chiappa et al., 2017; Babaeizadeh et al., 2017; Gemici et al., 2017; Denton and Fergus, 2018; Buesing et al., 2018; Doerr et al., 2018; Gregor and Besse, 2018), or com”
Temporal Difference Variational Auto-Encoder (2018, background): “, 2016), generative recall tasks (Gemici et al., 2017), and environment modeling (Oh et al.”