TartanAir: A Dataset to Push the Limits of Visual SLAM

By Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, Sebastian A. Scherer. 2020

720 citations recorded by the source corpus.

We present a challenging dataset, the TartanAir, for robot navigation tasks and more. The data is collected in photo-realistic simulation environments with the presence of moving objects, changing light and various weather conditions. By collecting data in simulations, we are able to obtain multi-modal sensor data and precise ground truth labels such as the stereo RGB image, depth image, segmentation, optical flow, camera poses, and LiDAR point cloud. We set up large numbers of environments with various styles and scenes, covering challenging viewpoints and diverse motion patterns that are difficult to achieve by using physical data collection platforms. In order to enable data collection at such a large scale, we develop an automatic pipeline, including mapping, trajectory sampling, data processing, and data verification. We evaluate the impact of various factors on visual SLAM algorithms using our data. The results of state-of-the-art algorithms reveal that the visual SLAM problem is far from solved. Methods that show good performance on established datasets such as KITTI do not perform well in more difficult scenarios. Although we use the simulation, our goal is to push the limits of Visual SLAM algorithms in the real world by providing a challenging benchmark for testing new methods, while also using a large diverse training data for learning-based methods. Our dataset is available at http://theairlab.org/tartanair-dataset.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In 3D Vision & Pose

Citations in the corpus

Vision Transformers for Dense Prediction (2021, methodology): “We construct a meta-dataset that includes the original datasets that were used in [32] (referred to as MIX 5 in that work) and extend it with with five additional datasets ([19, 45, 46, 48, 49]).”
Automated distance estimation for wildlife camera trapping (2022, methodology): “• TartanAir (Wang et al., 2020) is a photo-realistic synthetic dataset captured from persepctives of a flying drone and rendered with Unreal Engine.”
Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning (2024, methodology): “Follow ing the strategy in DPVO [35], we train our visual model on the TartanAir dataset [41] for 240000 iterations with a batch size of 1.”
Unifying Flow, Stereo and Depth Estimation (2022, methodology): “More specifically, we first finetune the Scene Flow pretrained model on the mixed Scene Flow [58], Tartan Air [109], Falling Things [110], CARLA HR-VS [107], CREStereo Dataset [62], InStereo2K [111] and Middlebury [15], [112] datasets for 100K iterations.”
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth (2023, methodology): “…we train on a mix of 12 datasets (M12) consisting of the 10 datasets used in [32]: HRWSI [46], Blend-edMVS [47], ReDWeb [45], DIML-Indoor [17], 3D Movies [33], MegaDepth [25], WSVD [41], TartanAir [43], ApolloScape [15] and IRS [42], plus 2 additional datasets: KITTI [29] and NYU Depth v2 [37].”
Foundation models in robotics: Applications, challenges, and the future (2023, positive): “For example, TartanAir [211], a dataset for robot navigation tasks, was collected in [212] with the presence of moving objects, changing light, and various weather conditions.”
MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion (2024, methodology): “During training, we sample the datasets asymmetrically to place extra weight on PointOdyssey (more dynamic, articulated objects) and less weight on TartanAir (good scene diversity but static) and Waymo (a highly specialized domain).”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “We train our network with a mixture of 14 datasets: Habitat [74], ARKitScenes [20], Blended MVS [112], MegaDepth [48], Static Scenes 3D [57], ScanNet++ [113], CO3D-v2 [67], Waymo [83], Map-free [5], WildRgb [2], VirtualKitti [12], Unreal4K [91], TartanAir [103] and an internal dataset.”
RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching (2021, background): “Sceneflow [23] Falling Things [36] Tartan Air [39] Sintel Stereo [3] ETH3D KITTI-15 Middlebury (Full)”
Continuous 3D Perception Model with Persistent State (2025, neutral): “Examples of our datasets include CO3Dv2 [60], ARK-itScenes [5], ScanNet++ [95], TartanAir [87], Waymo [75], MegaDepth [41], MapFree [3], DL3DV [44], and Dynam-icStereo [35].”
DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras (2021, methodology): “TartanAir [54] (Monocular & Stereo) The TartanAir dataset is a challenging synthetic benchmark for evaluating SLAM algorithms and was used as part of the ECCV 2020 SLAM competition.”
PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking (2023, background): “On the other hand, current rendering tools deliver such high photo-realism that the risk of a sim-to-real gap may be much smaller than seen in years past, making synthetic supervision increasingly viable [18, 61].”
Depth Anything 3: Recovering the Visual Space from Any Views (2025, positive): “In DA3, we substantially expand the training corpus to include: Hypersim [69], TartanAir [98], IRS [93], vKITTI2 [9], BlendedMVS [115], SPRING [56], MVSSynth [38], UnrealStereo4K [123], GTA-SfM [92], TauAgent [27], KenBurns [60], MatrixCity [50], EDEN [47], ReplicaGSO [82], UrbanSyn [32],…”
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (2025, positive): “Fast3R can be trained to achieve such results by finetuning a 16 static views checkpoint on the PointOdyssey [73] and TartanAir [62] datasets, consisting of 110 dynamic and 150 static scenes, respectively.”
GMFlow: Learning Optical Flow via Global Matching (2021, methodology): “, Virtual KITTI [5, 11], VIPER [33], REFRESH [26], AutoFlow [35] and TartanAir [44], they can be used to enhance Transformer’s generalization ability.”