Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs
By Yifeng Zhu, Jonathan Tremblay, Stan Birchfield, Yuke Zhu. 2020
153 citations recorded by the source corpus.
We present a visually grounded hierarchical planning algorithm for long-horizon manipulation tasks. Our algorithm offers a joint framework of neuro-symbolic task planning and low-level motion generation conditioned on the specified goal. At the core of our approach is a two-level scene graph representation, namely geometric scene graph and symbolic scene graph. This hierarchical representation serves as a structured, object-centric abstraction of manipulation scenes. Our model uses graph neural networks to process these scene graphs for predicting high-level task plans and low-level motions. We demonstrate that our method scales to long-horizon tasks and generalizes well to novel task goals. We validate our method in a kitchen storage task in both physical simulation and the real world. Experiments show that our method achieves over 70% success rate and nearly 90% of subgoal completion rate on the real robot while being four orders of magnitude faster in computation time compared to standard search-based task-and-motion planner. 1
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Kristen Grauman
- Jitendra Malik
- Li Fei-Fei
- Abhinav Kumar Gupta
- Dacheng Tao
- Trevor Darrell
- Andrew Zisserman
Citations in the corpus
- Continuous Scene Representations for Embodied AI (2022, methodology): “Our method is closer to the approaches that create scene graphs in embodied or physics based settings [6, 17, 31, 39, 67].”
- Open-vocabulary Queryable Scene Representations for Real World Planning (2022, background): “Recent progresses attempt to build a symbolic and geometric scene graph to facilitate task and motion planning [34].”
- ProgPrompt: Generating Situated Robot Task Plans using Large Language Models (2022, unknown): “[15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26].”