Argoverse: 3D Tracking and Forecasting With Rich Maps

By Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jagjeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan and 1 more. Computer Vision and Pattern Recognition, 2019

1,640 citations recorded by the source corpus.

We present Argoverse, a dataset designed to support autonomous vehicle perception tasks including 3D tracking and motion forecasting. Argoverse includes sensor data collected by a fleet of autonomous vehicles in Pittsburgh and Miami as well as 3D tracking annotations, 300k extracted interesting vehicle trajectories, and rich semantic maps. The sensor data consists of 360 degree images from 7 cameras with overlapping fields of view, forward-facing stereo imagery, 3D point clouds from long range LiDAR, and 6-DOF pose. Our 290km of mapped lanes contain rich geometric and semantic metadata which are not currently available in any public dataset. All data is released under a Creative Commons license at Argoverse.org. In baseline experiments, we use map information such as lane direction, driveable area, and ground height to improve the accuracy of 3D object tracking. We use 3D object tracking to mine for more than 300k interesting vehicle trajectories to create a trajectory forecasting benchmark. Motion forecasting experiments ranging in complexity from classical methods (k-NN) to LSTMs demonstrate that using detailed vector maps with lane-level information substantially reduces prediction error. Our tracking and forecasting experiments represent only a superficial exploration of the potential of rich maps in robotic perception. We hope that Argoverse will enable the research community to explore these problems in greater depth.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Environmental Engineering

View Paper: https://www.semanticscholar.org/paper/8e66c7e494476eb0dee846349df1bd705ceac6c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets (2021, positive): “Argoverse forecasting dataset [6] is a dataset with agent trajectories and highdefinition maps.”
Revealing Occlusions with 4D Neural Fields (2022, background): “Since point clouds are possible to collect at scale [7, 9], we believe they are an excellent source of data for learning to predict behind occlusions in video.”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, methodology): “In TableVII,methodsbasedonNGSIMI-80andUS-101highway drivingdatasets[194]arecomparedusingRMSE,whilewe useminADE,minFDE,MRtocomparemethodsbasedon Argoverse[77]inTableVIII,whichisrecordedunderurban conditionsandthepredictionhorizonis3seconds.”
HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction (2022, methodology): “We evaluate our prediction framework on the large-scale Argoverse motion forecasting dataset [10], which provides the trajectories of agents and the highdefinition map data.”
CoverNet: Multimodal Behavior Prediction Using Trajectory Sets (2019, background): “While there are encouraging new developments in public datasets [8, 20], there is no standard.”
Monocular depth estimation based on deep learning: An overview (2020, methodology): “he largest and most commonly used dataset for the sub-tasks in computer vision, like optical ﬂow [33], visual odometry [34], depth [35], object detection [36], semantic segmentation [37] and tracking [38], etc. It is also the commonest benchmark and the primary training dataset in the unsupervised a”
Unifying Flow, Stereo and Depth Estimation (2022, background): “…on 10 popular ﬂow/stereo/depth datasets (KITTI Flow [14], Sintel [13], Mid-dlebury [15], KITTI Stereo [14], ETH3D Stereo [32], Argoverse Stereo [33], ScanNet [34], SUN3D [35], RGBD-SLAM [36] and Scenes11 [37]), while being simpler and more efﬁcient in terms of model design and inference speed.”
Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D (2020, background): “ations from image data from multiple cameras builds on recent work in sensor fusion and monocular object detection. Large scale multi-modal datasets from Nutonomy [2], Lyft [13], Waymo [35], and Argo [3], have recently made full representation learning of the entire 360scene local to the ego vehicle”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, methodology): “Because the future is inherently stochastic and multi-modal, most models output a (weighted) set of trajectory hypotheses, and then a minimal error over the set (of constrained size) is reported (i.e. minADE [9]).”
Query-Centric Trajectory Prediction (2023, methodology): “(i) While the flourishing forecasting models have achieved impressive performance on trajectory prediction benchmarks [7,13,49], today’s most advanced architectures specialized for this task [37, 38, 46, 56] fail to process the heterogeneous traffic scenes efficiently.”
Canadian Adverse Driving Conditions dataset (2020, methodology): “More recently, the H3D Dataset [5], nuScenes Dataset [6], Argoverse Dataset [7], A*3D Dataset [8] and the Waymo”
Joint Monocular 3D Vehicle Detection and Tracking (2018, methodology): “Besides evaluating on synthetic data, we resort to Argoverse [6] and KITTI [13] tracking benchmarks to compare our model abilities.”
INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps (2019, methodology): “One includes motion data of surrounding entities from onboard LiDARs and front-view cameras, such as Argoverse [19] and HDD dataset [24].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “Other reviewed datasets such as [59], [77], [78], [81],”
nuScenes: A Multimodal Dataset for Autonomous Driving (2019, background): “The vectorized map expansion provides information on 11 semantic classes as shown in Figure 3, making it richer than the semantic maps of other datasets published since the original release [10, 45].”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, methodology): “, [54]. Maps have also been enriched to be leveraged for other vision tasks beyond localization such as road surface detection [55], [56], navigation [57], [58], object detection [59], [60], tracking [61] and forecasting [62]. This work uses a new form of map-based JOURNAL OF LATEX CLASS FILES, VOL.”
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation (2020, methodology): “Finally, we compare VectorNet with several baseline approaches [7] and some state-of-the-art methods on the Argoverse [7] test set.”