Learning Spatiotemporal Features with 3D Convolutional Networks

By Du Tran, Lubomir D. Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri. 2014

9,395 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Group Normalization (2018, methodology): “For example, the Fast/er and Mask R-CNN frameworks [12, 47, 18] use a batch size of 1 or 2 images because of higher resolution, where BN is “frozen” by transforming to a linear layer [20]; in video classification with 3D convolutions [60, 6], the presence of spatial-temporal features introduces a tr”
Non-local Neural Networks (2017, methodology): “To isolate the temporal effects of our non-local nets vs. 3D ConvNets, we construct a simple 2D baseline architecture in which the temporal dimension is trivially addressed (i.e., only by pooling).”
Late Temporal Modeling in 3D CNN Architectures with BERT for Action Recognition (2020, methodology): “The ﬁrst 3D CNN for AR is the C3D model [25].”
Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, background): “[6] trained 3D ConvNets [3] on the realistic and large-scale video datasets to learn both appearance and motion features with 3D convolution operations.”
VideoGPT: Video Generation using VQ-VAE and Transformers (2021, positive): “VideoGPT employs 3D convolutions and trans-posed convolutions (Tran et al., 2015) along with axial attention (Huang et al., 2019; Ho et al., 2019b) for the autoen-coder in VQ-VAE, learning a downsampled set of discrete latents from raw pixels of the video frames.”
Perceiver: General Perception with Iterative Attention (2021, background): “Second, it is possible to redesign architectural priors for data domains with different structures, such as videos (Tran et al., 2015) or audio (Ford et al., 2019), or for groups other than the group of linear translations (e.g. Cohen & Welling 2016; Bronstein et al. 2017; Esteves et al. 2018));…”
Joint Discovery of Object States and Manipulation Actions (2017, background): “Most of the prior work on action recognition has focused on designing features to describe time intervals of a video using motion and appearance [30, 39, 43, 44].”
Generating Videos with Scene Dynamics (2016, methodology): “We use spatio-temporal 3D convolutions to model videos [40], but we use fractionally strided convolutions [51] instead because we are interested in generation.”
Multiview Transformers for Video Recognition (2022, neutral): “Inspired by ViT, ViViT [3] and Timesformer [6] were the ﬁrst two works that successfully adopted a pure transformer architecture for video classiﬁcation, advancing the state of the art previously set by 3D CNNs. Multiscale processing in computer vision .”
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (2021, background): “, C3D [44], I3D [4, 13]) or optical flow to capture clip-level (or segment-level) motion information.”
ActBERT: Learning Global-Local Video-Text Representations (2020, background): “While supervised learning has been successful in a variety of computer vision tasks [18, 9, 40, 30], self-supervised representation learning from unlabeled data has attracted increasing attention in recent years [4, 28].”
Dynamic Neural Networks: A Survey (2021, background): “Although it has been observed that a low resolution might be sufficient for recognizing most ”easy” samples [27], conventional CNNs mostly process all the inputs with the same resolution, inducing considerable redundancy.”
The Kinetics Human Action Video Dataset (2017, methodology): “3D ConvNets [13, 21, 23] seem like a natural approach to video modeling.”
Action-Conditional Video Prediction using Deep Networks in Atari Games (2015, background): “, late-fusion or 3D convolution [35] can also be applied to this architecture).”
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering (2017, methodology): “Although our model is based on a sequence-to-sequence model [35] to achieve simplicity, it can be improved in different ways, such as adopting the concept of 3D convolution [34].”
Localizing Moments in Video with Natural Language (2017, methodology): “L, explained below). There are many choices for visual features. TALL (Gao et al., 2017) compares average fc 7 features (extracted from (Simonyan and Zisserman, 2014)) to features extracted with C3D (Tran et al., 2015) and LSTM features (Donahue et al., 2015). Surprisingly, C3D features only outperf”