Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing

By J. Lipton, Aidan J. Fay, Daniela Rus. 2017

214 citations recorded by the source corpus.

We demonstrate a low-cost telerobotic system that leverages commercial virtual reality (VR) technology and integrates it with existing robotics control infrastructure. The system runs on a commercial gaming engine using off-the-shelf VR hardware and can be deployed on multiple network architectures. The system is based on the homunculus model of mind wherein we embed the user in a VR control room. The control room allows for multiple sensor displays, and dynamic mapping between the user and robot. This dynamic mapping allows for selective engagement between the user and the robot. We compared our system with state-of-the-art automation algorithms and standard VR-based telepresence systems by performing a user study. The study showed that new users were faster and more accurate than the automation or a direct telepresence system. We also demonstrate that our system can be used for pick and place, assembly, and manufacturing tasks.

Subfields: Cognitive Neuroscience, Mechanical Engineering, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Roberto Mart'in-Mart'in
2. Silvio Savarese
3. Ajay Mandlekar
4. Ken Goldberg
5. Animesh Garg
6. Li Fei-Fei
7. Yuke Zhu
8. Daniela Rus

In Mechanical Engineering

1. Yuke Zhu
2. Ajay Mandlekar
3. Jonathan Booher
4. Pieter Abbeel
5. Li Fei-Fei
6. Silvio Savarese
7. Roberto Mart'in-Mart'in
8. Ken Goldberg

In Human-Computer Interaction

1. Ken Goldberg
2. Pieter Abbeel
3. Silvio Savarese
4. Animesh Garg
5. Li Fei-Fei
6. Ajay Mandlekar

Citations in the corpus

• Learning Multi-Arm Manipulation Through Collaborative Teleoperation (2020, methodology): “Furthermore, such systems can require sophisticated human-control interfaces [19, 22], such as Virtual Reality devices which are not widely available, consequently limiting the set of users that can participate in”
• ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation (2018, background): “…supervision can be provided through contemporary game interfaces such as a keyboard and mouse [26, 27], a video game controller [28], a 3D-mouse [29, 30], special purpose master-slave interfaces[31, 32], or through free-space positioning interfaces such as virtual reality (VR) [33, 34, 35].”
• Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation (2017, background): “In a non-learning setting, VR teleoperation has been recently explored for controlling humanoid robots [40], [41], [42], for simulated dexterous manipulation [43], and for communicating motion intent [44].”
• A research agenda for augmented and virtual reality in architecture, engineering and construction (2020, unknown): “VR interfaces for controlling equipment and robotic arms in manufacturing facilities have been investigated as well [98].”
• Helping Robots Learn: A Human-Robot Master-Apprentice Model Using Demonstrations via Virtual Reality Teleoperation (2020, methodology): “The homunculus framework for teleoperation via virtual reality presented in [49] was used to enable human supervision and remote control of a robot performing the grasping task.”