Transformers for One-Shot Visual Imitation

By Sudeep Dasari, Abhinav Kumar Gupta. 2020

111 citations recorded by the source corpus.

Humans are able to seamlessly visually imitate others, by inferring their intentions and using past experience to achieve the same end goal. In other words, we can parse complex semantic knowledge from raw video and efficiently translate that into concrete motor control. Is it possible to give a robot this same capability? Prior research in robot imitation learning has created agents which can acquire diverse skills from expert human operators. However, expanding these techniques to work with a single positive example during test time is still an open challenge. Apart from control, the difficulty stems from mismatches between the demonstrator and robot domains. For example, objects may be placed in different locations (e.g. kitchen layouts are different in every house). Additionally, the demonstration may come from an agent with different morphology and physical appearance (e.g. human), so one-to-one action correspondences are not available. This paper investigates techniques which allow robots to partially bridge these domain gaps, using their past experience. A neural network is trained to mimic ground truth robot actions given context video from another agent, and must generalize to unseen task instances when prompted with new videos during test time. We hypothesize that our policy representations must be both context driven and dynamics aware in order to perform these tasks. These assumptions are baked into the neural network using the Transformers attention mechanism and a self-supervised inverse dynamics loss. Finally, we experimentally determine that our method accomplishes a $\sim 2$x improvement in terms of task success rate over prior baselines in a suite of one-shot manipulation tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Decision Transformer: Reinforcement Learning via Sequence Modeling (2021, methodology): “Additionally, in imitation learning, some works have studied transformers as a replacement for LSTMs: Dasari and Gupta [70] study one-shot imitation learning, and Abramson et al. [71] combine language and image modalities for text-conditioned behavior generation.”
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “Prior works on imitation learning have shown one-shot or zero-shot generalization to new objects [1, 2, 3, 4, 5] and to new object goal configurations [6, 7].”
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition (2022, background): “, natural language processing (NLP) [27, 10, 82, 88], visual recognition [31, 79, 90, 63], dense prediction [83, 11, 98, 96, 86], Generative Adversarial Network (GAN) [52, 48], reinforcement learning (RL) [18, 16, 87], robotics [50, 25], and etc.”
Behavior Transformers: Cloning k modes with one stone (2022, methodology): “…in the second category implies that there is more multi-modality in the action distributions compared to action distributions of goal or reward conditioned datasets, which is the same reason a lot of work learning from multi-modal datasets try to learn a goal-conditioned model [35, 34, 52, 16].”
RVT: Robotic View Transformer for 3D Object Manipulation (2023, methodology): “Especially in object manipulation, transformer-based models with an attention mechanism can be utilized to extract features from sensory inputs to improve policy learning [47, 48, 49, 50, 51].”
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (2023, background): “…how robots can generalize to novel object instances [31, 34, 32, 35, 36], to tasks involving novel combinations of objects and skills [37, 38, 39, 40, 41], to new goals or language instructions [42, 43, 41, 44, 45, 46, 47], to tasks with novel semantic object categories [48, 49], and to unseen…”
VIMA: General Robot Manipulation with Multimodal Prompts (2022, methodology): “We follow prior works (Finn et al., 2017; Dasari & Gupta, 2020; Duan et al., 2017) to formulate the problem by giving one video demonstration (represented as key frames in prompts), then test the learned imitator’s ability to produce target trajectories.”
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation (2024, methodology): “Prior works also focus on multi-task or few-shot imitation learning, [25, 27, 30, 34, 46, 50, 88, 102], language-conditioned imitation learning [12, 47, 82, 83], imitation from play data [21, 57, 74, 89], using human videos [16, 24, 29, 60, 69, 80, 84, 96], and using task-specific structures [49,…”
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, methodology): “In robotics, Transformers have been applied to assistive teleop [41], legged locomotion [42], pathplanning [43, 44], imitation learning [45, 46], and grasping [47].”