CosyPose: Consistent multi-view multi-object 6D pose estimation

By Yann Labb'e, Justin Carpentier, Mathieu Aubry, Josef Sivic. European Conference on Computer Vision, 2020

546 citations recorded by the source corpus.

We introduce an approach for recovering the 6D pose of multiple known objects in a scene captured by a set of input images with unknown camera viewpoints. First, we present a single-view single-object 6D pose estimation method, which we use to generate 6D object pose hypotheses. Second, we develop a robust method for matching individual 6D object pose hypotheses across different input images in order to jointly estimate camera viewpoints and 6D poses of all objects in a single consistent scene. Our approach explicitly handles object symmetries, does not require depth measurements, is robust to missing or incorrect object hypotheses, and automatically recovers the number of objects in the scene. Third, we develop a method for global scene refinement given multiple object hypotheses and their correspondences across views. This is achieved by solving an object-level bundle adjustment problem that refines the poses of cameras and objects to minimize the reprojection error in all views. We demonstrate that the proposed method, dubbed CosyPose, outperforms current state-of-the-art results for single-view and multi-view 6D object pose estimation by a large margin on two challenging benchmarks: the YCB-Video and T-LESS datasets. Code and pre-trained models are available on the project webpage https://www.di.ens.fr/willow/research/cosypose/.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/486c76e7d8a4bcf3dce64bbc42551a0c73120fc9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (2023, background): “Instance-level pose estimation methods [19, 20, 30, 47] assume a textured CAD model is given for the object.”
DexYCB: A Benchmark for Capturing Hand Grasping of Objects (2021, background): “and CosyPose [19] (winner of the BOP challenge 2020).”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, unknown): “To account for symmetric objects, given ¯ R , the set of all possible ground-truth rotations under symmetry, we further extend our loss to a symmetry-aware formulation L R , sym = min”
Towards Unconstrained Joint Hand-Object Reconstruction From RGB Videos (2021, background): “3D pose estimation for hands [22, 31, 45, 56, 67] and objects [15, 26, 36, 39, 43, 49, 63, 64] have often been tackled in isolation [38].”
Instruction-driven history-aware policies for robotic manipulations (2022, background): “Deep neural networks [27, 28] have achieved impressive results in manipulation for single tasks [29], and recently led to more challenging setups such as multi-task learning [30, 31, 32, 33].”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, background): “Alternatively, along with the improvement of image quality and processing speed in rendering methods, abundant synthetic images can be automatically generated for network training [57], [60], [67], [68].”
BOP Challenge 2020 on 6D Object Localization (2020, methodology): “top-performing methods rely on RGB-D image channels, strong results have been achieved with RGB channels only. The challenge results and additional experiments with the top-performing CosyPose method [33] have shown the importance of PBR training images and of strong data augmentation for successful”