Learning Less is More - 6D Camera Localization via 3D Surface Regression

By Eric Brachmann, Carsten Rother. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

439 citations recorded by the source corpus.

Popular research areas like autonomous driving and augmented reality have renewed the interest in image-based camera localization. In this work, we address the task of predicting the 6D camera pose from a single RGB image in a given 3D environment. With the advent of neural networks, previous works have either learned the entire camera localization process, or multiple components of a camera localization pipeline. Our key contribution is to demonstrate and explain that learning a single component of this pipeline is sufficient. This component is a fully convolutional neural network for densely regressing so-called scene coordinates, defining the correspondence between the input image and the 3D scene space. The neural network is prepended to a new end-to-end trainable pipeline. Our system is efficient, highly accurate, robust in training, and exhibits outstanding generalization capabilities. It exceeds state-of-the-art consistently on indoor and outdoor datasets. Interestingly, our approach surpasses existing techniques even without utilizing a 3D model of the scene during training, since the network is able to discover 3D scene geometry automatically, solely from single-view constraints.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6ca43ff71b3c0039e7feb6e0b9fc7cf5d779e535

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Cross-View Semantic Segmentation for Sensing Surroundings (2019, methodology): “sk is still at pure image level without providing any information about the spatial structure of the surroundings explicitly. There are many attempts to extract spatial knowledge from the image input [8, 3, 6, 7], however, 1Code and demo video are available at the project page: https:// view-parsing”
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression (2019, methodology): “It also fails to outperform the simple DenseVLAD baseline on the Street scene, which is the largest and most complex scene in the Cambridge Landmarks dataset [10, 50].”
DUSt3R: Geometric 3D Vision Made Easy (2023, neutral): “Using a collection of pointmaps as shape representation is quite counter-intuitive for MVS, but its usage is widespread for Visual Localization tasks, either in scene-dependent optimization approaches [8, 9, 11] or scene-agnostic inference methods [96, 124, 159].”
Visual SLAM for Automated Driving: Exploring the Applications of Deep Learning (2018, methodology): “[8] used a sequence of less complex tasks.”
CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation (2019, methodology): “[4] used a fully convolutional network to predict 3D coordinates of scenes for 6-DoF camera pose estimation.”
A survey of deep learning techniques for autonomous driving (2019, methodology): “ion and Mapping (SLAM). For a survey on classical SLAM techniques, we refer the reader to [63]. Neural networks such as PoseNet [64], VLocNet++ [65], or the approaches introduced in [66], [67], [68], [69], or [70] are using image data to estimate the 3D pose of a camera in an End2End fashion. Scene ”
Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC (2020, methodology): “In previous work, we have shown that the RANSAC-based pose optimization can be made differentiable to allow for end-to-end training of a scene coordinate regression pipeline [14], [15].”
InLoc: Indoor Visual Localization with Dense Matching and View Synthesis (2019, methodology): “In contrast to other methods [10], [12], [13], [40], our approach does not need any scene-specific training.”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, positive): “In [60], [61], and [62], the authors introduce a novel differentiable way to apply RANSAC via sharing of hypotheses based on the predicted distribution.”