VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem

By R. Clark, Sen Wang, Hongkai Wen, Andrew Markham, Agathoniki Trigoni. AAAI Conference on Artificial Intelligence, 2017

401 citations recorded by the source corpus.

In this paper we present an on-manifold sequence-to-sequence learning approach to motion estimation using visual and inertial sensors. It is to the best of our knowledge the first end-to-end trainable method for visual-inertial odometry which performs fusion of the data at an intermediate feature-representation level. Our method has numerous advantages over traditional approaches. Specifically, it eliminates the need for tedious manual synchronization of the camera and IMU as well as eliminating the need for manual calibration between the IMU and camera. A further advantage is that our model naturally and elegantly incorporates domain specific information which significantly mitigates drift. We show that our approach is competitive with state-of-the-art traditional methods when accurate calibration data is available and can be trained to outperform them in the presence of calibration and synchronization errors.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/37832573b77b87f6ea0e6ecb52a8b3e6c4e05728

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM (2018, background): “Some particularly eye-catching pieces of work over the past year have focused on supervised and self-supervised training of surprisingly capable networks which are able to estimate visual odometry, depth and other quantities from video [10, 29, 30, 4, 32, 31].”
SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation (2018, methodology): “Following [35], [36], [37], we use the logarithm of a unit quaternion to parameterize the rotation in R(3) and do not require an added normalization constraint unlike previous works [38].”
NeurAll: Towards a Unified Model for Visual Perception in Automated Driving (2019, background): “Clark et al. [18] jointly pose visual and inertial data VO as a sequence to sequence learning problem.”
UnDeepVO: Monocular Visual Odometry Through Unsupervised Deep Learning (2017, positive): “Visual inertial odometry with deep learning was also developed in [13] and [14].”
Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning (2024, background): “Some end-to-end learning based VIO approaches have also been proposed [9, 14, 28].”
NeurAll: Towards a Unified Visual Perception Model for Automated Driving (2019, background): “[17] jointly pose visual and inertial data VO as a sequence to sequence learning problem.”
Active Neural Localization (2018, methodology): “…et al., 1960; Smith et al., 1990) geometry-based visual odometry methods (Nist´er et al., 2006) and most recently, learning-based visual odometry methods which learn to predict motion between consecutive frames using recurrent convolutional neural networks (Clark et al., 2017b; Wang et al., 2017).”
MapNet: An Allocentric Spatial Memory for Mapping Environments (2018, methodology): “DeepVO [24], VINet [25] and VidLoc [3] all employ Long-Short Term Memory (LSTM) on top of Convolutional Neural Networks (CNN) to predict pose changes, and are trained in a supervised manner.”