Deep learning from temporal coherence in video

By Hossein Mobahi, Ronan Collobert, Jason Aaron Edward Weston. 2009

367 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Representation & Self-Supervised Learning

Citations in the corpus

What makes ImageNet good for transfer learning? (2016, background): “The most well-known such criteria are image reconstruction [7, 38, 31, 29, 34, 23] (see [6] for a comprehensive overview) and feature slowness [48, 18].”
Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “As a result, and with the recent emergence of research video data sets [1, 59], videos have been investigated in the context of self-supervision (for example, [40, 64, 63, 28, 65, 73, 41, 51, 42, 3, 2]).”
Contrastive Multiview Coding (2019, neutral): “ed at time t, the ventral stream associates it with a neighbouring frame Method # of Views UCF-101 HMDB-51 Random - 48.2 19.5 ImageNet - 67.7 28.0 VGAN* [57] 2 52.1 - LT-Motion* [35] 2 53.0 - TempCoh [41] 1 45.4 15.9 Shufﬂe and Learn [39] 1 50.2 18.1 Geometry [17] 2 55.1 23.3 OPN [33] 1 56.3 22.1 ST”
Multi-task Self-Supervised Visual Learning (2017, background): “Another is to note that objects generally appear across many frames: thus, we can train features to remain invariant as a video progresses [11, 24, 42, 43, 47].”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, neutral): “One can also use the temporal structure in a video for self-supervised methods [21, 27, 41, 46, 47].”
Generating Videos with Scene Dynamics (2016, background): “Finally, this paper is related to a growing body of work that capitalizes on large amounts of unlabeled video for visual recognition tasks [18, 47, 37, 13, 24, 25, 3, 32, 26, 27, 19, 41, 42, 1].”
Representation Learning: A Review and New Perspectives (2012, background): “More recently, temporal coherence has been successfully exploited in deep architectures to model video (Mobahi et al., 2009).”
Learning to See by Moving (2015, methodology): “The quality of features learnt from this data were evaluated on four tasks - (1) Scene recognition on SUN [42] (section 5.1), (2) Visual odometery (section 5.3), (3) Keypoint matching (section 5.2) and (4) Object recognition on Imagenet [36] (section 5.4).”
Learning Image Representations Tied to Ego-Motion (2015, methodology): “We want to highlight the important distinctions between our objective and the “temporal coherence” objective of [22] for slow feature analysis.”
Colorization as a Proxy Task for Visual Understanding (2017, background): “It is also possible to train an embedding where temporally close frames are considered similar (using either pairs [25, 14, 15] or triplets [37]).”
Deep learning via semi-supervised embedding (2008, methodology): “See [28] and [18] for conference papers on this subject.”
Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning (2017, neutral): “2015; Jayaraman andGrauman 2015) or track content across frames (Wang and Gupta 2015; Mobahi et al. 2009; Goroshin et al. 2015).”
Visually Indicated Sounds (2015, methodology): “Several methods have also used temporal proximity as a supervisory signal [33, 17, 47, 46].”
Unsupervised Learning of Visual Representations Using Videos (2015, background): “Therefore, researchers have started focusing on learning feature representations using videos [11, 53, 27, 43, 56, 16, 48, 34, 45].”
UvA-DARE (Digital Academic Repository) Self-Supervised Video Representation Learning With Odd-One-Out Networks (critical): “As it can be seen from the results, Method UCF101-split1 HMDB51-split1 DrLim [17] 45.7 16.3 TempCoh [32] 45.4 15.9 Obj.”
The Challenges of Continuous Self-Supervised Learning (2022, background): “Self-supervised learning has a long history in computer vision [7, 35, 42, 50, 70, 71] aiming to learn representations of visual data by solving tasks that can be defined without human annotations.”
Unsupervised Feature Learning from Temporal Data (2015, background): “Consider a video sequence with T frames, if z t represents the feature vector extracted from the frame at time t then the slowness prior corresponds to minimizing =1 z t − z t − 1 (cid:107) p .”
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA (2016, methodology): “Kernel-based methods [12, 26] and deep learning methods [23, 27, 9] have been developed to extend this principle to the general nonlinear case.”