Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control
By Jonathan D. Chang, Nishanth Kumar, Sean P. Hastings, Aaron Gokaslan, Diego Romeres, Devesh K. Jha, Daniel Nikolaev Nikovski, George Dimitri Konidaris, Stefanie Tellex. 2019
11 citations recorded by the source corpus.
Robots need to learn skills that can not only generalize across similar problems but also be directed to a specific goal. Previous methods either train a new skill for every different goal or do not infer the specific target in the presence of multiple goals from visual data. We introduce an end-to-end method that represents targetable visuomotor skills as a goal-parameterized neural network policy. By training on an informative subset of available goals with the associated target parameters, we are able to learn a policy that can zero-shot generalize to previously unseen goals. We evaluate our method in a representative 2D simulation of a button-grid and on both button-pressing and peg-insertion tasks on two different physical arms. We demonstrate that our model trained on 33% of the possible goals is able to generalize to more than 90% of the targets in the scene for both simulation and robot experiments. We also successfully learn a mapping from target pixel coordinates to a robot policy to complete a specified goal.
Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Silvio Savarese
- Li Fei-Fei
- Jitendra Malik
- Xiaogang Wang
- Trevor Darrell
- Dacheng Tao
- Cordelia Schmid
In Control and Systems Engineering
- Li Fei-Fei
- Silvio Savarese
- Ajay Mandlekar
- Danfei Xu
- Roberto Mart'in-Mart'in
- Stefan Lee
- Mariano Phielipp
- Yuke Zhu
Citations in the corpus
- Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020, background): “, instructions [8], video demonstrations [41, 16, 10], or goal observations [6, 25].”
- Language-Conditioned Imitation Learning for Robot Manipulation Tasks (2020, background): “Neural approaches scale imitation learning [27, 4, 20, 1, 9] to high-dimensional spaces by enabling agents to learn task-specific feature representations.”