Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
By Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, N. Rudin, Arthur Allshire, Ankur Handa and 1 more. 2021
1,851 citations recorded by the source corpus.
Isaac Gym offers a high performance learning platform to train policies for wide variety of robotics tasks directly on GPU. Both physics simulation and the neural network policy training reside on GPU and communicate by directly passing data from physics buffers to PyTorch tensors without ever going through any CPU bottlenecks. This leads to blazing fast training times for complex robotics tasks on a single GPU with 2-3 orders of magnitude improvements compared to conventional RL training that uses a CPU based simulator and GPU for neural networks. We host the results and videos at \url{https://sites.google.com/view/isaacgym-nvidia} and isaac gym can be downloaded at \url{https://developer.nvidia.com/isaac-gym}.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Hardware and Architecture, ML Systems & Hardware, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Chelsea Finn
- Pieter Abbeel
- Yoshua Bengio
- Trevor Darrell
- Li Fei-Fei
- Mohit Bansal
- Jitendra Malik
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jitendra Malik
- Trevor Darrell
- Dacheng Tao
- Ming-Hsuan Yang
- Li Fei-Fei
- Xiaogang Wang
- Andrew Zisserman
In Hardware and Architecture
- Gabriel B. Margolis
- Jitendra Malik
- Ken Goldberg
- Dieter Fox
- Song-Chun Zhu
- Tete Xiao
- Pieter Abbeel
- Trevor Darrell
In ML Systems & Hardware
- Gabriel B. Margolis
- Ken Goldberg
- Dieter Fox
- Song-Chun Zhu
- Trevor Darrell
- Alexei A. Efros
- Zhiao Huang
- Jiajun Wu
In Robotics & Control
- Sergey Levine
- Chelsea Finn
- Pieter Abbeel
- Yuke Zhu
- Dieter Fox
- Ajay Mandlekar
- Jitendra Malik
- Stephen James
Citations in the corpus
- Learning to Learn with Generative Models of Neural Network Checkpoints (2022, methodology): “For our reinforcement learning (RL) experiments, we train policies for the Cartpole task using the IsaacGym simulator (Makoviychuk et al., 2021).”
- Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks (2023, background): “One key issue is that these robot simulation benchmarks often utilize physics simulators, e.g., MuJoCo [26] and IssacSim [13], which are significantly more difficult to extend than Minigrid and Miniworld.”
- Real-world humanoid locomotion with reinforcement learning (2023, methodology): “In our simulation environment, we use the Isaac Gym simulator ( 57, 58 ) to model the rigid-body and contact dynamics of the Digit humanoid robot.”
- DayDreamer: World Models for Physical Robot Learning (2022, methodology): “This enables massively parallel behavior learning with typical batch sizes of 16K on a single GPU, similar to specialized modern simulators (Makoviychuk et al., 2021).”
- MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023, methodology): “We show that MimicGen is not limited to just one simulation framework by applying it to high-precision tasks (requiring millimeter precision ) in Factory [51], a simulation framework built on top of Isaac Gym [52] to accurately simulate high-precision manipulation.”
- Robot Parkour Learning (2023, methodology): “We use IsaacGym [119] as the simulator to train all the policies.”
- ANYmal parkour: Learning agile navigation for quadrupedal robots (2023, methodology): “We train all the policies and collect the data for the perception module using the Isaac Gym simulator [42], where we deploy 4096 agents in parallel.”
- A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, neutral): “The work [199] provides an analysis of model-free regret-optimal best policy identification for online CMDPs. V.”
- A System for General In-Hand Object Re-Orientation (2021, methodology): “We use the simulated Shadow Hand [23] in NVIDIA Isaac Gym [24].”
- Real-World Robot Learning with Masked Visual Pre-training (2022, background): “The architecture of the controller closely 133 follows [52], i.”
- Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, neutral): “Embodiedperceptionrequires visualperceptionandreasoning,understandingthe3-Drelations within a scene, and predicting and performing complex tasks based on visual information.”
- CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms (2022, background): “The following table reports the final episodic returns obtained by the agent in Isaac Gym (Makoviychuk et al., 2021):”
- ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills (2023, methodology): “(5)For IsaacGym (Makoviychuk et al., 2021), we refer to its 4 manipulation tasks in the original paper: Franka Cube Stacking, Shadow Hand, Allegro Hand, and TriFinger.”
- Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior (2022, methodology): “We define our training environment in the Isaac Gym simulator [6, 25].”
- Evaluating Real-World Robot Manipulation Policies in Simulation (2024, positive): “That is, if one policy performs better than another in real-world 1] , higher is better) for assessing the correlation between policy performances in real-world and simulation, as well as the overall quality of simulated evaluation pipelines.”
- PhysDiff: Physics-Guided Human Motion Diffusion Model (2022, methodology): “We adopt IsaacGym [49] as the physics simulator for motion imitation.”
- Masked Visual Pre-training for Motor Control (2022, methodology): “We leverage the recent NVIDIA IsaacGym simulator (Makoviychuk et al., 2021) to build our benchmark.”
- Diffusion-based Generation, Optimization, and Planning in 3D Scenes (2023, methodology): “We test if a grasp is successful in IsaacGym [37] by applying external forces to the object and measuring the movement of the object.”