The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes

By Abhishek Patil, Srikanth Malla, Haiming Gang, Yi-Ting Chen. 2019

258 citations recorded by the source corpus.

3D multi-object detection and tracking are crucial for traffic scene understanding. However, the community pays less attention to these areas due to the lack of a standardized benchmark dataset to advance the field. Moreover, existing datasets (e.g., KITTI [1]) do not provide sufficient data and labels to tackle challenging scenes where highly interactive and occluded traffic participants are present. To address the issues, we present the Honda Research Institute 3D Dataset (H3D), a large-scale full-surround 3D multi-object detection and tracking dataset collected using a 3D LiDAR scanner. H3D comprises of 160 crowded and highly interactive traffic scenes with a total of 1 million labeled instances in 27,721 frames. With unique dataset size, rich annotations, and complex scenes, H3D is gathered to stimulate research on full-surround 3D multi-object detection and tracking. To effectively and efficiently annotate a large-scale 3D point cloud dataset, we propose a labeling methodology to speed up the overall annotation cycle. A standardized benchmark is created to evaluate full-surround 3D multi-object detection and tracking algorithms. 3D object detection and tracking algorithms are trained and tested on H3D. Finally, sources of errors are discussed for the development of future algorithms.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Cityscapes [270] X X Vehicle Berkley DeepDrive [271] X X Vehicle X X Mapillary [272] X X Vehicle X X Oxford RobotCar [49] X X Vehicle X X X KITTI [174] X X X X Vehicle H3D [273] X X X Vehicle ApolloScape [274] X X X X Vehicle nuScenes [175] X X X X Vehicle X X Udacity [275] X X X X Vehicle DDD17 [85”
nuScenes: A Multimodal Dataset for Autonomous Driving (2019, methodology): “The recent H3D dataset [61] includes 160 crowded scenes with a total of 1.”
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics (2019, methodology): “Unlike other filterbased MOT systems which define the state space of the filter in the 2D space [19] or bird’s eye view [20], we extend the state space of the objects to the 3D space, including 3D location, 3D size, 3D velocity and heading orientation.”
Deep Learning for 3D Point Clouds: A Survey (2019, unknown): “5K 18 Indoor RGB-D & Mesh H3D [26] 2019 160 8 27K 1.”
Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting (2023, methodology): “New sensor datasets for 3D object detection [4, 39, 34, 35, 21, 28, 16, 12, 36, 31] have led to influential detection methods such as anchor-based approaches like PointPillars [23], and more recent anchor-free approaches such as (1)This count includes private submissions not posted to the public lea”
Argoverse: 3D Tracking and Forecasting With Rich Maps (2019, background): “KITTI [10] and H3D [31] offers 3D bounding box and track annotations but does not provide a map and the camera field of view is frontal, rather than 360◦.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “Other reviewed datasets such as [59], [77], [78], [81],”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, methodology): “The modular pipeline starts by inputting raw sensory data to the perception module for obstacle detection [46] and localization via the localization module [3].”
One Million Scenes for Autonomous Driving: ONCE Dataset (2021, background): “The H3D dataset [31] provides point cloud data in 160 82 urban scenes.”
Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys (2023, background): “Some automobile manufacturers publish datasets collected by their vehicles, including H3D [128], A2D2 [129] and the Ford Dataset [130].”
Canadian Adverse Driving Conditions dataset (2020, methodology): “More recently, the H3D Dataset [5], nuScenes Dataset [6], Argoverse Dataset [7], A*3D Dataset [8] and the Waymo”
Scalability in Perception for Autonomous Driving: Waymo Open Dataset (2019, background): “ral Dataset [6] groups scenes recorded by multiple sensors, including a thermal imaging camera, by time slot, such as daytime, nighttime, dusk, and dawn. The Honda Research Institute 3D Dataset (H3D) [19] is a 3D object detection and tracking dataset that provides 3D LiDAR sensor readings recorded i”