Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
By Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, Shuran Song. Robotics: Science and Systems Conference, 2024
610 citations recorded by the source corpus.
We present Universal Manipulation Interface (UMI) -- a data collection and policy learning framework that allows direct skill transfer from in-the-wild human demonstrations to deployable robot policies. UMI employs hand-held grippers coupled with careful interface design to enable portable, low-cost, and information-rich data collection for challenging bimanual and dynamic manipulation demonstrations. To facilitate deployable policy learning, UMI incorporates a carefully designed policy interface with inference-time latency matching and a relative-trajectory action representation. The resulting learned policies are hardware-agnostic and deployable across multiple robot platforms. Equipped with these features, UMI framework unlocks new robot manipulation capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and long-horizon behaviors, by only changing the training data for each task. We demonstrate UMI's versatility and efficacy with comprehensive real-world experiments, where policies learned via UMI zero-shot generalize to novel environments and objects when trained on diverse human demonstrations. UMI's hardware and software system is open-sourced at https://umi-gripper.github.io.
Subfields: Computer Science Applications, Control and Systems Engineering, Mechanical Engineering
View Paper: https://www.semanticscholar.org/paper/40beef770a0f6b8cb2aa90587988b61080c40ba9
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Science Applications
In Control and Systems Engineering
In Mechanical Engineering
Citations in the corpus
- HumanPlus: Humanoid Shadowing and Imitation from Humans (2024, methodology): “Human data has been used extensively for robot learning, including for pre-training visual or intermediate representations or tasks [8, 28, 50, 53, 66, 79, 86, 99] leveraging Internet-scale data [18, 26, 27], and for imitation learning on in-domain human data [3, 12, 43, 47”
- A Survey on Vision--Language--Action Models for Embodied AI (2024, neutral): “UMI [216] proposes a method to mitigate these issues using hand-held grippers.”
- Open-TeleVision: Teleoperation with Immersive Active Visual Feedback (2024, neutral): “If we distinguish the existing work with sources of demonstrations, we can classify them into learning from real-robot expert data [14, 10, 12, 59, 60, 61, 62, 50, 63, 1, 64, 51, 65], learning from play data [66, 67, 68], and learning from human demonstrations [67, 69, 70, 71, 72, 73, 74, 56].”
- π0.5: a Vision-Language-Action Model with Open-World Generalization (2025, neutral): “…simple skills like picking up objects or opening drawers can be made to generalize simply by collecting robot data in a broader set of environments [14, 67, 28, 49, 64], it is challenging to apply the same approach to more complex, long-horizon tasks like cleaning up a kitchen, where achieving…”
- Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, neutral): “UMI[220]offersaframeworkforcollect-ingdynamic,bimanualdatausingahandheldgripper.”