A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry

By Zichao Zhang, Davide Scaramuzza. IEEE/RJS International Conference on Intelligent RObots and Systems, 2018

682 citations recorded by the source corpus.

In this tutorial, we provide principled methods to quantitatively evaluate the quality of an estimated trajectory from visual(-inertial) odometry (VO/VIO), which is the foundation of benchmarking the accuracy of different algorithms. First, we show how to determine the transformation type to use in trajectory alignment based on the specific sensing modality (i.e., monocular, stereo and visual-inertial). Second, we describe commonly used error metrics (i.e., the absolute trajectory error and the relative error) and their strengths and weaknesses. To make the methodology presented for VO/VIO applicable to other setups, we also generalize our formulation to any given sensing modality. To facilitate the reproducibility of related research, we publicly release our implementation of the methods described in this tutorial.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/b076c7b6e10ba80799a7bf28af06e178f1cc760b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

Events-To-Video: Bringing Modern Computer Vision to Event Cameras (2019, methodology): “We run a publicly available VIO evaluation toolbox [50] on raw trajectories provided by the authors of UltimateSLAM, which ensures that the trajectories estimated by all methods are evaluated in the exact same manner.”
NeRF-: Neural Radiance Fields Without Known Camera Parameters (2021, methodology): “…estimated using COLMAP, for evaluation, we first align the two trajectories globally with a Sim(3) transformation using an ATE toolbox [Zhang and Scaramuzza 2018], followed by a more fine-grained gradient-driven camera pose alignment by minimising the photometric error on the synthesised image,…”
NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior (2022, methodology): “In NeRFmm [8], the poses are first ini-tialised using Sim(3) alignment with an ATE toolbox [10].”
High Speed and High Dynamic Range Video with an Event Camera (2019, methodology): “We run a publicly available VIO evaluation toolbox [62] on raw trajectories provided by the authors of UltimateSLAM, which ensures”
Voxelized GICP for Fast and Accurate 3D Point Cloud Registration (2020, methodology): “Following [18], we calculated the absolute trajectory error (ATE) and relative error (RE) to evaluate the accuracy of the scan matching methods.”
R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package (2021, methodology): “To have more quantitative comparisons, we compute the relative rotation error (RPE) and relative translation error (RTE) [27] over all possible sub-sequences of length (50,100,150,...,300) meters are shown in Table.”