Geometric Consistency for Self-Supervised End-to-End Visual Odometry

By Ganesh Iyer, Krishna Murthy Jatavallabhula, Gunshi Gupta, K. Madhava Krishna, Liam Paull. 2018

53 citations recorded by the source corpus.

With the success of deep learning based approaches in tackling challenging problems in computer vision, a wide range of deep architectures have recently been proposed for the task of visual odometry (VO) estimation. Most of these proposed solutions rely on supervision, which requires the acquisition of precise ground-truth camera pose information, collected using expensive motion capture systems or high-precision IMU/GPS sensor rigs. In this work, we propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a self-supervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate. We also present a method of characterizing the uncertainty in VO estimates thus obtained. To evaluate our VO pipeline, we present exhaustive ablation studies that demonstrate the efficacy of end-to-end, self-supervised methodologies to train deep models for monocular VO. We show that leveraging concepts from geometry and incorporating them into the training of a recurrent neural network results in performance competitive to supervised deep VO methods.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

Karol Gregor

In Computer Vision and Pattern Recognition

Citations in the corpus

Shaping Belief States with Generative Environment Models for RL (2019, background): “Extracting a consistent high-level representation of the environment such as the bird’s eye view "map" from merely first-person views and actions in a completely unsupervised manner is a notoriously difficult task [12] and a lot of the success in addressing this problem is due to the injection of a ”
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey (2019, unknown): “Mehtod SubCategory Code Contribution VideoGAN [85] Generation 3 Forerunner of video generation with GAN MocoGAN [86] Generation 3 Decomposing motion and content for video generation with GAN TemporalGAN [144] Generation 3 Decomposing temporal and image generator for video generation Video Colorizati”