BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning

By Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, Trevor Darrell. 2018

3,030 citations recorded by the source corpus.

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Language-driven Semantic Segmentation (2022, methodology): “We finally train LSeg on a mix of 7 different datasets (Lambert et al., 2020), including ADE20K (Zhou et al., 2019), BDD (Yu et al., 2020), Cityscapes (Cordts et al., 2016), COCO-Panoptic (Lin et al., 2014; Caesar et al., 2018), IDD (Varma et al., 2019), Mapillary Vistas (Neuhold et al., 2017), and…”
ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes (2021, background): “Detection and Segmentation Datasets Many datasets for image segmentation have been proposed with the goal of densely recognizing general objects and “stuff” in image scenes like street view [7, 17, 61], natural scenes [9, 20, 35, 44,65], and indoor spaces [13,19,53].”
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming (2019, methodology): “We test this using BDD100k [Yu et al., 2018], a driving scene dataset with annotations for weather conditions.”
COMBO: Conservative Offline Model-Based Policy Optimization (2021, background): “This presents an attractive paradigm for data reuse and safe policy learning in many applications, such as healthcare [62], autonomous driving [65], robotics [25, 48], and personalized recommendation systems [59].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “Our findings contrast with previous findings that other tasks, such as object detection and segmentation, can suffer under the same types of shifts on the same dataset (Yu et al., 2020; Dai and Van Gool, 2018).”
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach (2021, background): “Off-policy data for AD includes, for example, public driving datasets [6, 22, 51].”
DSEC: A Stereo Event Camera Dataset for Driving Scenarios (2021, background): “Since then, a number of large scale datasets for semantic segmentation have been released [21]–[24].”
RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening (2021, methodology): “BDD-100K [63] is another real-world dataset that contains diverse urban driving scene images with the resolution of 1280 × 720.”
ByteTrack: Multi-Object Tracking by Associating Every Detection Box (2021, methodology): “We also test ByteTrack on HiEve [38] and BDD100K [78] datasets.”
Is Context-Aware CNN Ready for the Surroundings? Panoramic Semantic Segmentation in the Wild (2021, background): “…are a few wide-angle databases such as TorontoCity [51] and Wood-Scape [52], their scene variety and annotation density are far lower than pinhole benchmark datasets like Cityscapes [5], Mapillary Vistas [6], IDD [53] and BDD [54], which suppose rich ontologies for producing robust segmenters.”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, background): “Creating a dataset for motion forecasting is complicated by the fact that the distribution of real world data is highly imbalanced [4, 18, 31, 37]; in the common case, vehicles drive straight at a constant velocity.”
UniDepth: Universal Monocular Metric Depth Estimation (2024, methodology): “The training dataset utilized is the ensemble of Argoverse2 [43], Waymo [39], Driv-ingStereo [45], Cityscapes [6], BDD100K [49], Mapillary-PSD [1], A2D2 [18], ScanNet [7], and Taskonomy [51].”
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions (2024, methodology): “The videos of ShareGPT4Video are collected from various sources [12, 66, 48, 18, 49, 43], employed with a Semantic-based Data Filtering strategy to mitigate content homogeneity among these videos.”