Convolutional Learning of Spatio-temporal Features

By Graham W. Taylor, Rob Fergus, Yann LeCun, C. Bregler. 2010

702 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Spectral Networks and Locally Connected Networks on Graphs (2013, background): “id structure (in 1, 2 and 3 dimensions), and the data to be studied in those coordinates has translational equivariance/invariance with respect to this grid. Speech [11], images [14, 20, 22] or video [23, 18] are prominent examples that fall into this category. On a regular grid, a CNN is able to ex”
Representation Learning: A Review and New Perspectives (2012, methodology): “Convolutional versions of the RBM or its extensions have also been developed (Desjardins and Bengio, 2008; Lee et al., 2009a; Taylor et al., 2010) as well as a probabilistic max-pooling operation built into Convolutional Deep Networks (Lee et al., 2009a,b; Krizhevsky, 2010).”
The Kinetics Human Action Video Dataset (2017, methodology): “3D ConvNets [13, 21, 23] seem like a natural approach to video modeling.”
Least Squares Generative Adversarial Networks (2016, methodology): “the given data. Restricted Boltzmann Machines (RBMs), one type of deep generative models, are the basis of many other hierarchical models, and they have been used to model the distributions of images [15] and documents [16]. Deep Belief Networks (DBNs) [17] and Deep Boltzmann Machines (DBMs) [5] are”
Unsupervised Learning of Visual Representations Using Videos (2015, background): “Therefore, researchers have started focusing on learning feature representations using videos [11, 53, 27, 43, 56, 16, 48, 34, 45].”
Describing Videos by Exploiting Temporal Structure (2015, background): “While other work has explored 3D Deep Networks for video [33, 14, 16, 28] our particular approach differs in a number of ways from prior work in that it is based on CNNs as opposed to other 3D deep architectures and we focus on pre-training the model on a number of widely used action recognition…”
SlowFast Networks for Video Recognition (2018, background): “3D ConvNets [48, 49, 5] extend 2D image models [32, 45, 47, 24] to the spatiotemporal domain, handling both spatial and temporal dimensions similarly.”
Spatiotemporal Residual Networks for Video Action Recognition (2016, methodology): “As actions can be understood as spatiotemporal objects, researchers have investigated carrying spatial recognition principles over to the temporal domain by learning local spatiotemporal ﬁlters [13, 25, 26].”
Differential Recurrent Neural Networks for Action Recognition (2015, neutral): “Ji et al. [14] 90.20 Taylor et al. [28] 90.0 Laptev et al. [17] 91.80 Dollar et al. [5] 81.20 3DCNN [2] 89.40”
3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks (2014, background): “On the other hand, the past few years have seen a resurgence of research in the design of deep neural networks, and impressive progresses were made on learning image features from raw data [7, 27, 12, 11, 17, 5].”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, neutral): “[126] extended the Gated RBM (GRBM) [91] to convolutional GRBM (convGRBM) that shares weights at all locations in an image and inference is performed through convolution.”
Action recognition with trajectory-pooled deep-convolutional descriptors (2015, methodology): “The second type of representations are the deep-learned features, and typical methods include Convolutional RBMs [29], 3D ConvNets [9], Deep ConvNets [12], and TwoStream ConvNets [24].”
C3D: Generic Features for Video Analysis (2014, background): “3D ConvNets were proposed for human action recognition [20, 46] and for medical image segmentation [19, 48].”
Convolutional Two-Stream Network Fusion for Video Action Recognition (2016, methodology): “As with other areas of computer vision, recent work has concentrated on applying Convolutional Neural Networks (ConvNets) to this task, with progress over a number of strands: learning local spatiotemporal filters [11, 28, 30]), incorporating optical flow snippets [22], and modelling more extended t”
Actions ~ Transformations (2015, background): “Learned representations with deep learning have recently produced state of the art results in action recognition [17, 21, 39, 52, 48, 49, 28, 8, 58].”
FlowNet: Learning Optical Flow with Convolutional Networks (2015, methodology): “[33] approach the task with factored gated restricted Boltzmann machines.”