Learning Multilevel Distributed Representations for High-Dimensional Sequences

By Ilya Sutskever, Geoffrey E. Hinton. 2007

247 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Temporal Generative Adversarial Nets with Singular Value Clipping (2016, background): “Temporal RBMs have also been proposed to explicitly capture the temporal correlations in videos [40, 38, 37].”
Evolution of Cartesian Genetic Programs for Development of Learning Neural Architecture (2011, background): “Deep belief nets are extensively used for generating and recognizing images, video sequences, and motion-capture data (Hinton et al., 2006; Sutskever and Hinton, 2006; Taylor et al., 2006).”
Deep Temporal Sigmoid Belief Networks for Sequence Modeling (2015, background): “Experimental results on bouncing balls, polyphonic music, motion capture, and text streams show that the proposed approach achieves state-of-the-art predictive performance, and has the capacity to synthesize various sequences.”
State-of-the-art review on deep learning in medical imaging. (2019, background): “The important applications of DBN has been in the generation and recognition of images (15, 16), video sequences (17) and motion capture data (18).”
Modeling Human Motion Using Binary Latent Variables (2006, methodology): “We can model temporal dependencies by tr eating the visible variables in the previous time slice as additional fixed inputs [10].”
Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks (2016, methodology): “A pivotal work to model highdimensional sequences was based on using Temporal Restricted Boltzman Machines (Sutskever and Hinton 2007; Sutskever, Hinton, and Taylor 2009; Mittelman et al. 2014).”
Learning Deep Architectures for AI (2007, methodology): “Instead, [180] propose to use a mean-field filtering approximation of the hidden sequence posterior.”
A generative restricted Boltzmann machine based method for high-dimensional motion data modeling (2015, unknown): “Hinton [27] proposes the CD algorithm to approximate (cid:104) v i h j (cid:105) model by using a reconstructed sample, which gives the following weight change: Given the visible layer, the hidden units are independent with each other, so sampling the states of hidden units can be performed in paral”
Learning multiple layers of representation. (2007, background): “This review has focused on static images, but restricted Boltzmann machines can also be applied to high-dimensional sequential data such as video sequences [34] or the joint angles of a walking person [35].”
Markov Decision Process for Video Generation (2019, methodology): “One of the first networkbased models for motion dynamics used a temporal extension of Restricted Boltzmann Machines (RBMs) [24, 27] with a focus on resolving the intractable inference [25].”
A Survey on Context Learning (2017, methodology): “This results in effective adaptations: the Temporal Restricted Boltzmann Machine (TRBM) [11].”
Modular deep belief networks that do not forget (2011, methodology): “While DBNs (and the related approach of stacked autoencoders) have been successfully applied to many tasks [1, 2, 5–7], most approaches rely on training data that are sampled from a stationary distribution.”
The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour (2016, background): “…improvements to this general blueprint are possible (e.g. convolutional feature extraction (LeCun & Bengio, 1995) and extension to sequential data (Sutskever & Hinton, 2007)); see (Hinton, 2009) for a concise overview, (Bengio, 2009) for a tutorial introduction and (Schmidhuber, 2015) for an…”
Two Distributed-State Models For Generating High-Dimensional Time Series (2011, background): “Keywords: unsupervised learning, restricted Boltzmann machines, time series, generative models, motion capture”
Hierarchical Convolutional Deep Learning in Computer Vision (2013, methodology): “145 9.2.3 Factored Third-order Input-Output Temporal Restricted Boltz- mann Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 9.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 9.3.1 2D facial expression transfer . . . . . . . . . . . . . .”
ApproxDBN: Approximate Computing for Discriminative Deep Belief Networks (2017, methodology): “On the other hand, stochastic Deep Belief Networks (DBNs) have been used for generating and recognizing images [9], video sequences [10], and motion-capture data [11].”