Anticipating Visual Representations from Unlabeled Video

By Carl Vondrick, Hamed Pirsiavash, Antonio Torralba. 2015

525 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Scaling Autoregressive Video Models (2019, background): “Generative modeling of video holds promise for applications such as content creation, forecasting, transfer learning and model-based reinforcement learning (Srivastava et al., 2015; Carl Vondrick, 2016; Oh et al., 2015; Kaiser et al., 2019).”
VideoBERT: A Joint Model for Video and Language Representation Learning (2019, methodology): “Thus our model is not a generative model of pixels, but it is a generative model of features derived from pixels, which is an approach that has been used in other work ( e.g ., [29]).”
Learning Video Representations from Textual Web Supervision (2020, methodology): “For example, prior approaches have successfully lever-aged supervision from clip and frame ordering [47, 16, 38, 66, 68, 33], geometry [17, 29], motion [53, 37], colorization [62], cycle consistency [14, 64], video prediction [44, 42, 61, 60, 63], tempo [69], and in-video consistency [22, 54].”
End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, background): “Some of these tasks include: temporal ordering of video clips or frames [24, 46, 54, 86], predicting geometric transformations [38], maximizing the mutual information of multiple views [74], predicting motion and appearance [78], predicting the future, the past or a portion of masked input in the fe”
Generating Videos with Scene Dynamics (2016, background): “We instead leverage large amounts of unlabeled video for generation.”
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “In the research community, activity forecasting has been often formulated as an extension of action classification, tracking, or feature generation: Given a sequence of image frames, they directly predict a few categorized actions [124, 125, 126], human locations [127], or image features [128] to be”
Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “Previous work considers future localization [113, 120, 174, 230], action anticipation [76, 77, 86, 118, 127, 219], next active object prediction [20, 74], future event prediction [149,167], and future frame prediction [145,146,153,215,218,227].”
From Recognition to Cognition: Visual Commonsense Reasoning (2018, neutral): “Commonsense Reasoning Our task unifies work involving reasoning about commonsense phenomena, such as physics [54, 84], social interactions [2, 77, 12, 27], procedure understanding [91, 3] and predicting what might happen next in a video [74, 17, 92, 78, 18, 64, 85].”
SoundNet: Learning Sound Representations from Unlabeled Video (2016, background): “Cross-Modal Learning and Unlabeled Video: Our approach is broadly inspired by efforts to model cross-modal relations [24, 14, 7, 26] and works that leverage large amounts of unlabeled video [25, 41, 8, 40, 39].”
Stochastic Prediction of Multi-Agent Interactions from Partial Observations (2019, methodology): “( Vondrick et al., 2016a) train a conditional CNN to predict future feature vectors, and several papers (e.g., (Lerer et al., 2016; Mottaghi et al., 2016; Wu et al., 2017; van Steenkiste et al., 2018; Fragkiadaki et al., 2016; Watters et al., 2017)) predict future object states.”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, background): “For example, Vondrick et al. (2016a) introduced a framework that capitalizes on P.”
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset (2018, methodology): “Similarly to [47], we set τ a = 1 s .”
Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks (2016, positive): “Thus, in our work, rather than modeling a motion prior as a context-free distribution, we propose to model the conditional motion distribution of future frames given an input image by incorporating a series of low-and high-level image cues.”
Asynchronous Temporal Fields for Action Recognition (2016, background): “With the advent of deep learning, learning representations from data has been extensively studied [14, 15, 44, 57, 52, 53, 24, 7, 61, 55, 40, 3].”