Self-supervised Spatiotemporal Feature Learning by Video Geometric Transformations

By Longlong Jing, Yingli Tian. 2018

82 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Learning Video Representations from Textual Web Supervision (2020, methodology): “For example, prior approaches have successfully lever-aged supervision from clip and frame ordering [47, 16, 38, 66, 68, 33], geometry [17, 29], motion [53, 37], colorization [62], cycle consistency [14, 64], video prediction [44, 42, 61, 60, 63], tempo [69], and in-video consistency [22, 54].”
Self-Supervised MultiModal Versatile Networks (2020, background): “single modality), such as predicting the relative position of patches [18, 59], colorization [95], predicting orientation [27] or invariance to transformation [20, 36].”
End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, neutral): “Some of these tasks include: temporal ordering of video clips or frames [24, 46, 54, 86], predicting geometric transformations [38], maximizing the mutual information of multiple views [74], predicting motion and appearance [78], predicting the future, the past or a portion of masked input in the fe”
Video Representation Learning by Dense Predictive Coding (2019, methodology): “6 3D-RotNet [14] (112× 112) 3D-ResNet18-full (33.”
Audio-Visual Instance Discrimination with Cross-Modal Agreement (2020, background): “7 3D-RotNet [37] Kinetics400 16×112(2) 62.”
Spatiotemporal Contrastive Video Representation Learning (2020, background): “Aside from predicting the future, other common approaches include sorting frames or video clips [42, 72, 39, 18] along the temporal dimension and learning from proxy tasks like rotation [37].”
Audiovisual SlowFast Networks for Video Recognition (2020, methodology): “Results: Self-supervised Learning method inputs #param FLOPs pretrain UCF HMDB Shufﬂe&Learn [49, 69] V 58.3M N/A K600 26.5 12.6 3D-RotNet [35, 69] V 33.6M N/A K600 47.7 24.8 CBT [69] A ∗ +V N/A N/A K600 We only train the the last fc layer after self-supervised pretraining on Kinetics-400…”
Video Representation Learning with Visual Tempo Consistency (2020, unknown): “4 RotNet3D [26] 3D-ResNet18 16 62.”
Speech2Action: Cross-Modal Supervision for Action Recognition (2020, unknown): “4 3DRotNet [18]⋆ S3D-G (RGB) Kinetics† 40.”
Self-supervised Video Representation Learning for Correspondence Flow (2019, unknown): “Of more direct relevance to our training framework are selfsupervised frameworks that use video data [1, 7, 9, 10, 15, 16, 17, 18, 19, 21, 24, 26, 29, 38, 42, 46].”
Multi-modal Self-Supervision from Generalized Data Transformations (2020, background): “These include predicting geometric transformations [24], jigsaw puzzles [26] and performing noise contrastive retrieval [17].”
Self-supervised Co-training for Video Representation Learning (2020, unknown): “motion [1, 14, 31], co-occurrence [30], jigsaw [37], rotation [33], speed prediction [6, 17, 62], future prediction [25, 26, 60], or by temporal coherence [40, 41, 61, 63].”
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey (2019, methodology): “Mehtod SubCategory Code Contribution VideoGAN [85] Generation 3 Forerunner of video generation with GAN MocoGAN [86] Generation 3 Decomposing motion and content for video generation with GAN TemporalGAN [144] Generation 3 Decomposing temporal and image generator for video generation Video Colorizati”