Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC
By Eric Brachmann, Carsten Rother. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
318 citations recorded by the source corpus.
We describe a learning-based system that estimates the camera position and orientation from a single input image relative to a known environment. The system is flexible w.r.t. the amount of information available at test and at training time, catering to different applications. Input images can be RGB-D or RGB, and a 3D model of the environment can be utilized for training but is not necessary. In the minimal case, our system requires only RGB images and ground truth poses at training time, and it requires only a single RGB image at test time. The framework consists of a deep neural network and fully differentiable pose optimization. The neural network predicts so called scene coordinates, i.e., dense correspondences between the input image and 3D scene space of the environment. The pose optimization implements robust fitting of pose parameters using differentiable RANSAC (DSAC) to facilitate end-to-end training. The system, an extension of DSAC++ and referred to as DSAC*, achieves state-of-the-art accuracy on various public datasets for RGB-based re-localization, and competitive accuracy for RGB-D based re-localization.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/1ed542999771fadbba41ef12bfeea4a67eee35f2
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Marc Pollefeys
- Paul-Edouard Sarlin
- Torsten Sattler
- Carl Toft
- Fredrik Kahl
- Lars Hammarstrand
- Vincent Lepetit
- Viktor Larsson
In Computer Vision and Pattern Recognition
- Marc Pollefeys
- Torsten Sattler
- Luc Van Gool
- Paul-Edouard Sarlin
- Jitendra Malik
- Andrew Zisserman
- Andrea Vedaldi
- Ming-Hsuan Yang
Citations in the corpus
- DUSt3R: Geometric 3D Vision Made Easy (2023, unknown): “Using a collection of pointmaps as shape representation is quite counter-intuitive for MVS, but its usage is widespread for Visual Localization tasks, either in scene-dependent optimization approaches [8, 9, 11] or scene-agnostic inference methods [96, 124, 159].”
- Back to the Feature: Learning Robust Camera Localization from Pixels to Pose (2021, background): “Those trained per scene include 3D coordinate regression networks DSAC* RGB [13] and HACNet [46], and CAMNet [24], which regresses a relative pose following image retrieval.”