MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations

By Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj S. Narang, Linxi (Jim) Fan, Yuke Zhu, Dieter Fox. Conference on Robot Learning, 2023

398 citations recorded by the source corpus.

Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents. However, the demonstrations can be extremely costly and time-consuming to collect. We introduce MimicGen, a system for automatically synthesizing large-scale, rich datasets from only a small number of human demonstrations by adapting them to new contexts. We use MimicGen to generate over 50K demonstrations across 18 tasks with diverse scene configurations, object instances, and robot arms from just ~200 human demonstrations. We show that robot agents can be effectively trained on this generated dataset by imitation learning to achieve strong performance in long-horizon and high-precision tasks, such as multi-part assembly and coffee preparation, across broad initial state distributions. We further demonstrate that the effectiveness and utility of MimicGen data compare favorably to collecting additional human demonstrations, making it a powerful and economical approach towards scaling up robot learning. Datasets, simulation environments, videos, and more at https://mimicgen.github.io .

View Paper: https://www.semanticscholar.org/paper/54f107c7cdffc43490ba18e3e4313a8b4dff4950

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Ted Xiao
2. O. Team
3. Homer Rich Walke
4. Dibya Ghosh
5. Sudeep Dasari
6. Oier Mees
7. Dorsa Sadigh
8. Joey Hejna

Citations in the corpus

• Octo: An Open-Source Generalist Robot Policy (2024, background): “In recent years, there have also been multiple efforts for building robot manipulation datasets of increasing scale and diversity, either collected via scripted and autonomous policies [19, 41, 42, 12, 71, 30] or human teleoperation [59, 60, 25, 90, 39, 10, 26, 6, 77, 63, 79].”
• RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots (2024, methodology): “To augment our datasets, we extend MimicGen [34] to generate 100K additional trajectories for our atomic tasks.”