Implicit 3D Orientation Learning for 6D Object Detection from RGB Images

By Martin Sundermeyer, Zoltán-Csaba Márton, Maximilian Durner, Manuel Brucker, Rudolph Triebel. European Conference on Computer Vision, 2018

600 citations recorded by the source corpus.

We propose a real-time RGB-based pipeline for object detection and 6D pose estimation. Our novel 3D orientation estimation is based on a variant of the Denoising Autoencoder that is trained on simulated views of a 3D model using Domain Randomization.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3fc26991557d28ff720edef2d284ef2663e1e530

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation (2019, methodology): “Instead of a subset of the test sequences in [10], full test images are used to compare with the state of the art [28].”
Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model Using Deep Non-Rigid Structure From Motion (2019, background): “Over the past few years rapid progress has been made in the direction of disentangling the latent space of deep models into dimensions that account for generic factors of variation, such as identity and low-dimensional transformations [8, 52, 28, 51, 39], or even non-rigid, dense deformations from a”
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data (2018, unknown): “[24] use DR for object detection and 6D pose estimation, achieving competitive results compared to pose estimation using real data.”
HOnnotate: A Method for 3D Annotation of Hand and Object Poses (2019, background): “Methods for 3D pose estimation of rigid objects and hands from monocular images have made signiﬁcant progress recently, thanks to the development of Deep Learning, and the creation of large datasets or the use of synthetic images for training [32, 35, 46, 55, 71, 73].”
BOP Challenge 2020 on 6D Object Localization (2020, methodology): “One of the key ingredients of CosyPose is a strong data augmentation technique similar to [48].”
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation (2018, methodology): “To avoid this problem, [38, 33, 23, 35] discretize the rotation space and cast the 3D rotation estimation into a classification task.”
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, unknown): “Holistic approaches [27, 15, 18, 68, 33, 63, 58, 60, 44, 54] directly output pose parameters from RGB images.”
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects (2018, background): “For alternate approaches to related problems, see [33, 34, 35, 36, 37].”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “To address this problem, some approaches use post-reﬁnement procedure [26, 50] to reﬁne the pose iteratively, others discrete the rotation space and simplify it to be a classiﬁcation problem [49, 43, 45].”
DeepIM: Deep Iterative Matching for 6D Pose Estimation (2018, methodology): “A recent work (Sundermeyer et al., 2018) uses an autoen-coder to map the object in the image to a vector and search for the most similar vector in a pre-generated codebook for pose estimation.”
Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, unknown): “Most methods [22, 28, 42] try to limit the amount of required annotations by relying on synthetic data.”
iNeRF: Inverting Neural Radiance Fields for Pose Estimation (2020, methodology): “Train data With Real Pose Labels Without Real Pose Labels Methods DPOD[51] DeepIM[14] AAE[39] MHP [20] DPOD[51] Self6D[48] Ours-init Ours-iNeRF Ape 53.”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, background): “Domain Randomization (DR) is the common strategy in domain generalization [64, 53, 49, 63, 65, 59, 70, 77, 50] especially when a single source domain is available [64, 70, 77, 50].”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, background): “Pose estimation via intermediate representation is sensitive to outliers in predictions, which are introduced by occlusion and cluttered backgrounds [37, 32, 40].”
DeceptionNet: Network-Driven Domain Randomization (2019, methodology): “Building on top of this idea, the methods of [34, 45] extended this to the RGB domain.”
Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data (2019, methodology): “DR has also been utilized to do object detection and 6D pose estimation [49, 36, 45].”
Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation (2020, background): “[26] generates a latent code for the input image and search for its nearest neighbor in the codebook.”