DeepIM: Deep Iterative Matching for 6D Pose Estimation

By Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang, Dieter Fox. International Journal of Computer Vision, 2018

791 citations recorded by the source corpus.

Estimating 6D poses of objects from images is an important problem in various applications such as robot manipulation and virtual reality. While direct regression of images to object poses has limited accuracy, matching rendered images of an object against the input image can produce accurate results. In this work, we propose a novel deep neural network for 6D pose matching named DeepIM. Given an initial pose estimation, our network is able to iteratively refine the pose by matching the rendered image against the observed image. The network is trained to predict a relative pose transformation using a disentangled representation of 3D location and 3D orientation and an iterative training process. Experiments on two commonly used benchmarks for 6D pose estimation demonstrate that DeepIM achieves large improvements over state-of-the-art methods. We furthermore show that DeepIM is able to match previously unseen objects.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/fd577c0b62d4d2d2d088566ab7e87bfd2d5a43f1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation (2019, background): “e textured 3D model of an object, which is made by a 3D scanning device, e.g., BigBIRD Object Scanning Rig [36], and provided by a dataset to render synthetic images for training [15,29] or reﬁnement [19,22]. Thus, the quality of texture in the 3D model should be sufﬁcient to render visually correct”
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion (2019, methodology): “It is widely adopted by both classical methods [4, 8, 37] and recent learning-based approaches [17, 31, 34].”
CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, methodology): “Fourth, instead of ﬁxing focal lengths to 1 during training as in [10], we use focal lengths of the camera equivalent to the cropped images.”
iNeRF: Inverting Neural Radiance Fields for Pose Estimation (2020, methodology): “We evaluate our method with the ADD [9] and ADD-S [14] metrics.”
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (2023, methodology): “Orthogonal to single-frame object pose estimation, pose tracking methods [8, 28, 34, 37, 51, 57, 61, 66] leverage temporal cues to enable more efficient, smooth and accurate pose estimation on a video sequence.”
CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation (2019, positive): “We follow [13] to split the dataset.”
KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects (2019, methodology): “The first type of technique aligns the observed RGB images with rendered CAD model images [5, 11], or aligns the observed 3D point clouds with 3D CAD model point clouds with algorithms such as ICP [28], or renders mixed reality data from 3D CAD models as additional training data [29].”
Unseen Object Instance Segmentation for Robotic Environments (2020, methodology): “Many methods employ some level of rendering randomization [59], [60], [61], [9], [62], [63], including lighting conditions and textures.”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, methodology): “Besides the reweighting strategy, some recent works propose to use deep learning-based refiners to boost the pose estimation performance [19, 26, 44].”
DPOD: 6D Pose Object Detector and Refiner (2019, methodology): “It compares the following ADD scores: 1) ADD reported in the original DeepIM paper [18], which used PoseCNN [33] to predict initial poses, 2) ADD if DeepIM is applied to poses predicted by our detector, 3) ADD if poses predicted by the proposed detector are reﬁned with the proposed reﬁnement.”
Deep Template-based Object Instance Detection (2021, background): “Additionally, recent 6-DOF trackers achieve generic instance tracking using simple viewpoint renders from a 3D model [9, 24, 27].”
Online Replanning in Belief Space for Partially Observable Task and Motion Problems (2019, methodology): “ tasks. The stow spam, inspect drawer, switch drawers, and cook spam videos show the robot solving the tasks in table I We used PoseCNN [37] to detect several YCB objects [38] in the scene and DeepIM [39] to reﬁne the estimates of their poses. Finally, we used DART [27] to track the robot arm, door ”
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, neutral): “PoseCNN DeepIM [68, 33] PVNet[46] CDPN[34] DPOD[70] PointFusion[69] DenseFusion[65] G2LNet[7] PVN3D[17] Our FFB6D”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “To address this problem, some approaches use post-reﬁnement procedure [26, 50] to reﬁne the pose iteratively, others discrete the rotation space and simplify it to be a classiﬁcation problem [49, 43, 45].”
BOP Challenge 2020 on 6D Object Localization (2020, methodology): “The importance of a refinement stage has been demonstrated also by other methods – top nine methods applied ICP or an RGB-based refiner, similar to DeepIM [34] (column “Refine” in Tab.”
ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape (2018, methodology): “In contrast, [40, 24, 30] directly encode the 6D pose.”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, positive): “Therefore, common choices are unit quaternions [51], [55], [69], log quaternions [83], or Lie algebra-based vectors [53].”
DexYCB: A Benchmark for Capturing Hand Grasping of Objects (2021, methodology): “For DeepIM and PoseRBPF, we also include their RGB-D variants.”
Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, positive): “DeepIM [24] shows generalization to unseen objects by iteratively matching rendered images of an object against observed ones.”