Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd

By Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, Tae-Kyun Kim. 2015

268 citations recorded by the source corpus.

Object detection and 6D pose estimation in the crowd (scenes with multiple object instances, severe foreground occlusions and background distractors), has become an important problem in many rapidly evolving technological areas such as robotics and augmented reality. Single shotbased 6D pose estimators with manually designed features are still unable to tackle the above challenges, motivating the research towards unsupervised feature learning and next-best-view estimation. In this work, we present a complete framework for both single shot-based 6D object pose estimation and next-best-view prediction based on Hough Forests, the state of the art object pose estimator that performs classification and regression jointly. Rather than using manually designed features we a) propose an unsupervised feature learnt from depth-invariant patches using a Sparse Autoencoder and b) offer an extensive evaluation of various state of the art features. Furthermore, taking advantage of the clustering performed in the leaf nodes of Hough Forests, we learn to estimate the reduction of uncertainty in other views, formulating the problem of selecting the next-best-view. To further improve pose estimation, we propose an improved joint registration and hypotheses verification module as a final refinement step to reject false detections. We provide two additional challenging datasets inspired from realistic scenarios to extensively evaluate the state of the art and our framework. One is related to domestic environments and the other depicts a bin-picking scenario mostly found in industrial settings. We show that our framework significantly outperforms state of the art both on public and on our datasets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

The limits and potentials of deep learning for robotics (2018, background): “Some work is undertaken in the area of next-best view-point prediction to improve object detection [5], [25], [73], [130] or path planning for exploration on a mobile robot [14], but a more holistic approach to active scene understanding is still missing from current research.”
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, unknown): “Instead, dense correspondence approaches [36, 59, 14, 2, 41, 28, 12, 34, 66, 6, 6, 21, 4] find the correspondence between image pixels and mesh vertexes and recover poses within Perspective-nPoint (PnP) manners.”
T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects (2017, background): “global description relying primarily on intensity edges and depth cues [20, 24, 54, 5, 14, 21, 27].”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “They either use random forest [3, 32] or CNNs [23, 9, 27, 35, 51] to extract feature and predict the corresponding 3D object coordinates for each pixel and then vote for the ﬁnal pose results.”
BOP Challenge 2020 on 6D Object Localization (2020, unknown): “LM [18] 15 – 50000 – 18273 3000 18273 3000 LM-O [3] ∗ 8 – 50000 – 1214 200 9038 1445 T-LESS [23] ∗ 30 37584 50000 – 10080 1000 67308 6423 ITODD [9] ∗ 28 – 50000 54 721 721 3041 3041 HB [30] ∗ 33 – 50000 4420 13000 300 67542 1630 YCB-V [53] ∗ 21 113198 50000 – 20738 900 98547 4123 RU-APC [45] 14 – – ”
Learning Where to Look: Data-Driven Viewpoint Set Selection for 3D Scenes (2017, background): “Next-best view prediction has been addressed in the context of various problems in robotics and computer vision: 3D reconstruction [17], volumetric occupancy prediction [39], and object pose estimation [7] among others.”
BOP: Benchmark for 6D Object Pose Estimation (2018, unknown): “LM/LM-O [14,1] IC-MI [34] IC-BIN [7] T-LESS [16] RU-APC [28] TUD-L - new TYO-L - new”
A dataset for developing and benchmarking active vision (2017, background): “[27] gives a system for object detection, pose estimation, and next best view prediction.”
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation (2018, methodology): “To utilize the powerful CNNs, [18, 7] densely sample image patches and use networks to extract features for the latter voting.”
Deep Template-based Object Instance Detection (2021, background): “Despite this, efforts have been made to capture such real datasets [3, 13, 16, 33, 7, 34] and to combine them together [17].”
CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, background): “Another approach [29] uses local RGBD patches to generate object hypotheses and find the best view of a scene.”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, unknown): “Finally, we compare our method withstate-of-the-artmethodsontheBOPbenchmark[34],which containssevencoredatasetsincludingLM-O[86],YCB-V[51], T-LESS[93],TUD-L[94],IC-BIN[95],ITODD[96]and HB[97].”
6-DoF object pose from semantic keypoints (2017, background): “ they are inadequate for addressing textureless objects and their performance is susceptible to scene clutter. An alternative to sparse discriminative keypoints is offered by dense methods [9], [10], [11], [12], where every pixel or patch is voting for the object pose. These approaches are also appl”