Temporal Shift Module for Efficient Video Understanding

By Ji Lin, Chuang Gan, Song Han. 2018

221 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Weight Standardization (2019, methodology): “The reimplementation is different from the original TSM [21]: we use models pre-trained on Ima-Method Then, we compare the performances when different normalization methods are used in training TSM. Table 6 shows the top-1/5 accuracies of TSM when trained with GN, GN+WS, BN and BN+WS.”
Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video (2020, background): “[29] proposed the Temporal Shift Module (TSM), a component which facilitates information exchange among neighboring frames without the introduction of extra parameters in the network.”
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection (2021, methodology): “We use the segment-based input frame sampling [21,52] (split each video into segments, and sample one frame from each segment to form a clip).”
Only Time Can Tell: Discovering Temporal Data for Temporal Modeling (2019, methodology): “Recently, TSM [24] augments 2D CNN with temporal shift modules to achieve temporal modeling.”
A Multigrid Method for Efficiently Training Video Models (2019, methodology): “The model is pre-trained on Kinetics-400 following prior work [29].”
Grouped Spatial-Temporal Aggregation for Efficient Action Recognition (2019, background): “TSM [15] replace temporal filters with shift modules.”
Adversarial Cross-Domain Action Recognition with Co-Attention (2019, background): “Temporal Shifting Module (Lin, Gan, and Han 2018) shifts feature channels along the temporal dimension to model temporal information efﬁciently.”
Collaborative Distillation in the Parameter and Spectrum Domains for Video Action Recognition (2020, background): “Lin et al. (Lin, Gan, and Han 2018) shift part of the channels along the temporal dimension to enable the temporal information transmission between neighboring frames.”
Context-Gated Convolution (2019, methodology): “For the action recognition task, we adapt three baselines to evaluate the effectiveness of our CGC: TSN (Wang et al., 2016), P3D-A (Qiu et al., 2017) (details are in appendix), and TSM (Lin et al., 2018).”
Few-Shot Video Classification via Temporal Alignment (2019, background): “…schemes can be mainly divided into two categories: dense sampling (randomly cropping out T consecutive frames from the original full-length video and then dropping every other frame) [37, 4, 42, 45] and sparse sampling (samples distribute uniformly along the temporal dimension) [41, 48, 46, 21].”
STM: SpatioTemporal and Motion Encoding for Action Recognition (2019, methodology): “…video clips which shows basic human interactions with daily ResNet-101 71.4 - Disentangling [39] BNInception 71.5 89.9 R(2+1)D RGB [28] ResNet-34 72.0 90.0 R(2+1)D Two-Stream [28] 73.9 90.9 TSM [19] ResNet-50 72.5 90.7 TSN RGB [33] BNInception 69.1 88.7 TSN Two-Stream [33] 73.9 91. objects.”
Lower Dimensional Kernels for Video Discriminators (2019, methodology): “The Temporal Shift Module (TSM) (Lin et al., 2018), entirely forgoes 3D and/or 1D kernels.”