Modeling Deep Temporal Dependencies with Recurrent "Grammar Cells"

By Vincent Michalski, Roland Memisevic, Kishore Reddy Konda. 2014

120 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions (2018, background): “Purely unsupervised approaches for state-to-state prediction (Agrawal et al., 2016; Lerer et al., 2016; Michalski et al., 2014; Sutskever et al., 2009) that use raw visual inputs as state-descriptions have yet to rival these capabilities.”
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision (2016, background): “Learning from transformed sensory data has gained attention [12, 5, 15, 13, 23, 6, 24] in recent years.”
Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis (2015, background): “The Predictive Gating Pyramid (PGP) [19] is developed for time series prediction, and is able to learn image transformations including shifts and rotation over multiple time steps.”
Learning Visual Predictive Models of Physics for Playing Billiards (2015, background): “The early development of smooth pursuit suggests that it is important for further development of visual and motor faculties of a human infant.”
Multi-view 3D Models from Single Images with a Convolutional Network (2015, background): “These often involve multiplicative interactions, for example gated RBMs [2], gated autoencoder [3] or Disentangling Boltzmann Machines [4].”
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense (2017, neutral): “See, for example, [18, 21] and references therein.”
"What Happens If..." Learning to Predict the Effect of Forces in Images (2016, background): “[23] model temporal dependencies of a sequence and predict multiple steps in the future.”
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning (2016, methodology): “Our work here builds on previous work in next-frame video prediction [26, 28, 29, 38, 48], but we take particular inspiration from the concept of “predictive coding” from the neuroscience literature [2, 5, 7, 9, 11, 21, 25, 32, 39, 40, 47, 49].”
Slow and Steady Feature Analysis: Higher Order Temporal Coherence in Video (2015, neutral): “Another class of methods learns transformations [29, 28, 33].”
Learning to Generate Long-term Future via Hierarchical Prediction (2017, background): “Early work on future frame prediction focused on small patches containing simple predictable motions (Sutskever et al., 2009; Michalski et al., 2014; Mittelman et al., 2014) and motions in real videos (Ranzato et al., 2014; Srivas-tava et al., 2015).”
Unsupervised Learning of Visual Structure using Predictive Generative Networks (2015, background): “Recently, Michalski et al. (2014) proposed a predictive training scheme where a transformation is ﬁrst inferred between two frames and is then applied again to obtain a prediction of a third frame.”
A Review on Deep Learning Techniques for Video Prediction (2020, background): “Inspired by these models, Michalski et al. proposed the Predictive Gating Pyramid (PGP) [159] consisting on a recurrent pyramid of stacked GAEs.”
Time-series modeling with undecimated fully convolutional neural networks (2015, background): “ [10], however, training RNNs can be a difﬁcult task due to the vanishing and exploding gradients problems [16,23]. There have been many works that focussed on extending the modeling capacity of RNNs [22,26], and alleviating the vanishing and exploding gradients problem [25]. The long short term mem”
Unsupervised Learning of Visual Representations using Videos (2015, background): “Michalski et al. (2014) extend this idea by modelling not just pairs but longer images sequences.”
Learning image representations equivariant to ego-motion (2015, background): “The limited prior work on unsupervised feature learning with video [20] [22] [19] [7] learns only passively from observed scene dynamics, uninformed by explicit motor sensory cues.”
SimVP: Simpler yet Better Video Prediction (2022, background): “Recurrent Grammar Cells [41] stacks multiple gated autoencoders in a recurrent pyramid structure.”
Long-Term Image Boundary Prediction (2016, background): “Other works (Sutskever, Hinton, and Taylor 2009; Michalski, Memisevic, and Konda 2014) focus on deterministic bouncing ball sequences, but their dataset is limited in size and resolution and generalization with respect to the number of balls and their velocities is not considered.”
Long-Term Image Boundary Extrapolation (2016, background): “Works of [22, 16] predict frames of videos of bouncing balls, but their dataset is very limited in size and resolution.”
Unsupervised Learning of Video Representations using LSTMs (2015, background): “This work was extended recently by Michalski et al. (2014) to model longer sequences.”