Unsupervised Learning of Visual Representations using Videos

By Nitish Srivastava. 2015

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Masked Autoencoders Are Scalable Vision Learners (2021, background): “Self-supervised learning approaches have seen significant interest in computer vision, often focusing on different pretext tasks for pre-training [15, 61, 42, 70, 45, 17].”
Momentum Contrast for Unsupervised Visual Representation Learning (2019, background): “The networks f q and f k can be identical [29, 59, 63], partially shared [46, 36, 2], or different [56]. x q encoder encoder q encoder k q Conceptual comparison of three contrastive loss mechanisms (empirical comparisons are in Figure 3 and Table 3).”
Rethinking ImageNet Pre-Training (2018, background): “, for selfsupervised learning [5, 47, 33, 32]), as now we learn that even random initialization could produce excellent results.”
What makes ImageNet good for transfer learning? (2016, background): “Attempted self-supervised tasks include predictions of ego-motion [1, 20], spatial context [11, 33, 30], temporal context [45], and even color [51] and sound [32].”
Self-Supervised Learning of Video-Induced Visual Invariances (2019, methodology): “These tracked patches have been used to learn image representations [62].”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, methodology): “For example, camera motion [1, 32], tracking [61], text proximity [41], and other cues [11, 62, 30, 60, 29] can be used to ﬁnd examples of features that should be ‘similar’ or predictable from one another, whereas negative examples are usually sampled randomly.”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “Our findings also suggest a bright future for unsupervised or self-supervised [10, 43] representation learning approaches.”
Representation Learning with Contrastive Predictive Coding (2018, methodology): “In Computer Vision [28] use a triplet loss on tracked video patches so that patches from the same object at different timesteps are more similar to each other than to random patches.”
Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces (2017, neutral): “TextTopicNet (Wiki) 67 44 39 53 20 49 68 42 43 33 41 35 70 57 82 30 31 39 65 41 Sound [25] 69 45 38 56 16 47 65 45 41 25 37 28 74 61 85 26 39 32 69 38 Texton-CNN 65 35 28 46 11 31 63 30 41 17 28 23 64 51 74 9 19 33 54 30 K-means 61 31 27 49 9 27 58 34 36 12 25 21 64 38 70 18 14 25 51 25 Motion [41] ”
Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018, methodology): “For videos, self-supervision strategies include: leveraging temporal continuity via tracking [44, 45], predicting future [42], or preserving the equivariance of egomotion [13, 50, 30].”
Unsupervised Representation Learning by Predicting Image Rotations (2018, neutral): “ImageNet labels from (Bojanowski & Joulin, 2017) 59.7 59.7 Random from (Noroozi & Favaro, 2016) 27.1 12.0 Tracking Wang & Gupta (2015) 38.8 29.8 Context (Doersch et al., 2015) 45.6 30.4 Colorization (Zhang et al., 2016a) 40.7 35.2 Jigsaw Puzzles (Noroozi & Favaro, 2016) 45.3 34.6 BIGAN (Donahue et…”
Learning Features by Watching Objects Move (2016, background): “Wang and Gupta [43] train a ConvNet to distinguish between pairs of tracked patches in a single video, and pairs of patches from different videos.”
Context Encoders: Feature Learning by Inpainting (2016, background): “Context encoders are competitive with concurrent self-supervised feature learning methods [7, 39] and significantly outperform autoencoders and Agrawal et al.”
Unsupervised Visual Representation Learning by Context Prediction (2015, background): “This kind of temporal coherence has a long history in visual learning literature [18, 56], and contemporaneous work shows strong improvements on modern detection datasets [54].”
Training Region-Based Object Detectors with Online Hard Example Mining (2016, background): “There is recent work [22, 27, 33] concurrent to our own that selects hard examples for training deep networks.”
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (2018, methodology): “In Table 5, we explore how well our perceptual task correlates to semantic tasks on the PASCAL dataset [15], using results summarized in [63], including additional selfsupervised methods [1, 43, 12, 56, 62, 41].”
Self-Supervised Learning of Pretext-Invariant Representations (2019, background): “For video data, these pretext tasks learn representations by ordering video frames [1, 18, 34, 41, 51, 79, 83], tracking [62, 77], or using cross-modal signals like audio [2, 3, 19, 36, 60, 61].”