Time-Contrastive Networks: Self-Supervised Learning from Video

By Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine. 2017

943 citations recorded by the source corpus.

We propose a self-supervised approach for learning representations and robotic behaviors entirely from unlabeled videos recorded from multiple viewpoints, and study how this representation can be used in two robotic imitation settings: imitating object interactions from videos of humans, and imitating human poses. Imitation of human behavior requires a viewpoint-invariant representation that captures the relationships between end-effectors (hands or robot grippers) and the environment, object attributes, and body pose. We train our representations using a triplet loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. This signal causes our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. We demonstrate that this representation can be used by a robot to directly mimic human poses without an explicit correspondence, and that it can be used as a reward function within a reinforcement learning algorithm. While representations are learned from an unlabeled collection of task-related videos, robot behaviors such as pouring are learned by watching a single 3rd-person demonstration by a human. Reward functions obtained by following the human demonstrations under the learned representation enable efficient reinforcement learning that is practical for real-world robotic systems. Video results, open-source code and dataset are available at sermanet.github.io/imitate

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Representation & Self-Supervised Learning

Citations in the corpus

Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “Timecontrastive networks [53] consider two views of the same action to learn invariant representations.”
Learning Representations by Maximizing Mutual Information Across Views (2019, background): “Self-supervised learning is gaining popularity across the NLP, vision, and robotics communities – e.g., [Devlin et al., 2019, Logeswaran and Lee, 2018, Sermanet et al., 2017, Dwibedi et al., 2018].”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “, 2016), and robotic manipulation (Pinto & Gupta, 2016; Pinto et al., 2016; Sermanet et al., 2018).”
S4L: Self-Supervised Semi-Supervised Learning (2019, background): “Self-supervised techniques have a variety of applications in a broad range of computer vision topics [14, 35, 7, 29, 34].”
Contrastive Multiview Coding (2019, background): “Time contrastive networks [67] use a triplet loss framework to learn representations from aligned video sequences of the same scene, taken by different video cameras.”
A critical analysis of self-supervision, or what we can learn from a single image (2019, background): “Typical pretext tasks are based on temporal-context (Misra et al., 2016; Wei et al., 2018; Lee et al., 2017; Sermanet et al., 2018), spatio-temporal cues (Isola et al., 2015; Gao et al., 2016; Wang et al., 2017), foreground-background segmentation via video segmentation (Pathak et al., 2017),…”
Revisiting Self-Supervised Visual Representation Learning (2019, background): “Similarly, when learning representation from videos, one can either make use of the synchronized cross-modality stream of audio, video, and potentially subtitles [35, 39, 23, 44], or of the consistency in the temporal dimension [41].”
What makes for good views for contrastive learning (2020, background): “…luminance and chrominance [65], applying different random crops and data augmentations [73, 8, 4, 26, 75, 62], pasting an object into different backgrounds [79], using different timesteps within a video sequence [50, 80, 57, 25, 24], or using different patches within a single image [32, 50, 30].”
Self-labelling via simultaneous clustering and representation learning (2019, background): “Typical pretext tasks are based on temporal-context (Misra et al., 2016; Wei et al., 2018; Lee et al., 2017; Sermanet et al., 2018), spatio-temporal cues (Isola et al., 2015; Gao et al., 2016; Wang et al., 2017), foreground-background separation via video segmentation (Pathak et al., 2017),…”
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, background): “to account for potential domain shifts between observations [16, 39, 40].”
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation (2020, background): “Recently, the success of deep learning has garnered increased interest in learning to solve perception and manipulation tasks in an end-to-end framework [11, 12, 28, 29, 30].”
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels (2020, background): “, 2018) or by employing auxiliary losses that provide additional supervision (Yarats et al., 2019; Srinivas et al., 2020; Sermanet et al., 2018; Dwibedi et al., 2018).”
Self-Supervised Correspondence in Visuomotor Policy Learning (2019, methodology): “A more generalizable direction may be offered by unsupervised methods of obtaining reward signals [2], [17], [18].”
Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, methodology): “Another method learns the correspondence in an unsupervised fashion with a small number of human supervision [ Sermanet et al. , 2018 ] .”