FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects

By Bowen Wen, Wei Yang, Jan Kautz, Stanley T. Birchfield. Computer Vision and Pattern Recognition, 2023

682 citations recorded by the source corpus.

We present FoundationPose, a unified foundation model for 6D object pose estimation and tracking, supporting both model-based and model-free setups. Our approach can be instantly applied at test-time to a novel object without finetuning, as long as its CAD model is given, or a small number of reference images are captured. Thanks to the unified framework, the downstream pose estimation modules are the same in both setups, with a neural implicit representation used for efficient novel view synthesis when no CAD model is available. Strong generalizability is achieved via large-scale synthetic training, aided by a large language model (LLM), a novel transformer-based architecture, and contrastive learning formulation. Extensive evaluation on multiple public datasets involving challenging scenarios and objects indicate our unified approach outperforms existing methods specialized for each task by a large margin. In addition, it even achieves comparable results to instance-level methods despite the reduced assumptions. Project page: https://nvlabs.github.io/FoundationPose/

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/dc4c9ae8c0cfc08ff6392aff69b0fd170da398a4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Fei-Fei Li
  2. Chen Wang
  3. Ruohan Zhang
  4. Yunzhu Li
  5. Wenlong Huang

Citations in the corpus