R3M: A Universal Visual Representation for Robot Manipulation

By Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, Abhi Gupta. 2022

933 citations recorded by the source corpus.

We study how visual representations pre-trained on diverse human video data can enable data-efficient learning of downstream robotic manipulation tasks. Concretely, we pre-train a visual representation using the Ego4D human video dataset using a combination of time-contrastive learning, video-language alignment, and an L1 penalty to encourage sparse and compact representations. The resulting representation, R3M, can be used as a frozen perception module for downstream policy learning. Across a suite of 12 simulated robot manipulation tasks, we find that R3M improves task success by over 20% compared to training from scratch and by over 10% compared to state-of-the-art visual representations like CLIP and MoCo. Furthermore, R3M enables a Franka Emika Panda arm to learn a range of manipulation tasks in a real, cluttered apartment given just 20 demonstrations. Code and pre-trained models are available at https://tinyurl.com/robotr3m.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, methodology): “One approach to grounding language models in interaction is by learning downstream networks with pre-trained LLM representations [38, 22, 21, 39, 40, 41, 42, 43].”
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots (2024, background): “Utilizing passive human demonstrations, previous works learn task cost functions [37, 8, 1, 21], affordance functions [2], dense object descriptors [40, 24, 39], action correspondences [33, 28], and pre-trained visual representations [23, 48].”
VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training (2022, methodology): “We follow the training and evaluation protocol of (Nair et al., 2022) and consider 12 tasks combined from FrankaKitchen, MetaWorld (Yu et al., 2020), and Adroit (Rajeswaran et al., 2017), 3 camera views for each task, and 3 demonstration dataset sizes, and report the aggregate average maximum…”
Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models (2023, neutral): “Distinct from our approach for utilizing generative models, other works design representation learning objectives for vision-language pretraining for control [49, 42, 35, 4], but these methods still need to utilize limited amounts of data from the target task to learn a policy.”
Diffusion policy: Visuomotor policy learning via action diffusion (2023, positive): “We tested Diffusion Policy with frozen pretrained vision encoders (ImageNet (Deng et al., 2009) and R3M (Nair et al., 2022)), as seen in Table 6.”
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models (2023, neutral): “Language grounding has been studied extensively both in terms of intelligent agents [19–22] and of robotics [23, 6, 24, 25, 5, 7, 26], where language can be used as a tool for compositional goal specification [5, 27–33], semantic anchor for training multimodal representation [12, 34, 35], or as an i”
CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation (2024, positive): “In recent years, there has been a surge of interest in robotic control models equipped with visual capabilities [7, 8, 15, 30, 34, 45, 48, 58, 60, 62, 67, 69].”
A Survey on Vision--Language--Action Models for Embodied AI (2024, positive): “1) Components of VLA: Pretraining: Value-Implicit Pre-training (VIP) [40] capitalizes on the temporal sequences present in videos, distinguishing itself from R3M [39] by attracting both the initial and target frames while simultaneously repelling successive frames.”
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, methodology): “Another line of work tackles data inefficiency by using pre-trained image representations [16, 35, 36] to bootstrap BC.”
Foundation models in robotics: Applications, challenges, and the future (2023, positive): “…[29] Language-AssistedReinforcementLearning e.g. Adaptive Agent (AdA) [30], Palo et al. [15] Language-ImageGoal-Conditioned ValueLearning e.g. R3M [31], SayCan [32], Inner Monologue [33], VoxPoser [34], Mahmoudieh et al. [35], VIP [36], LIV [37], LOREL [38] High-LevelTaskPlanning e.g. NL2TL [39],…”
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models (2022, methodology): “In robotics, they have been used as representations for perception [37, 42], as task representation for language [24, 30], or as planners [1, 22].”
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought (2023, methodology): “For embodied control tasks, we compare our model with R3M[12], which is the state-of-the-art method in these two benchmarks, and an ablation version called ’BLIP-2[Ego4D]’, which has the same structure and same amount of parameters as EmbodiedGPT, Table 2: Ablation on the closed-loop spans from…”
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? (2023, methodology): “WeﬁndthatR3Menablesdataefﬁcientimitationlearningacrossseveralsimulatedandreal-worldrobotmanipulationtasks.”
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation (2023, methodology): “We compare with four baseline methods: MCIL (Lynch & Sermanet, 2020), RT-1 (Brohan et al., 2022), HULC (Mees et al., 2022b), and a multi-task version of R3M (Nair et al., 2022).”
Real-World Robot Learning with Masked Visual Pre-training (2022, background): “[30] 60 explore pre-training visual representations using time contrastive learning and language descriptions 61 from human annotators.”