YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video

By Esteban Real, Jonathon Shlens, Stefano Mazzocchi, Xin Pan, Vincent Vanhoucke. 2017

577 citations recorded by the source corpus.

We introduce a new large-scale data set of video URLs with densely-sampled object bounding box annotations called YouTube-BoundingBoxes (YT-BB). The data set consists of approximately 380,000 video segments about 19s long, automatically selected to feature objects in natural settings without editing or post-processing, with a recording quality often akin to that of a hand-held cell phone camera. The objects represent a subset of the COCO [32] label set. All video segments were human-annotated with high-precision classification labels and bounding boxes at 1 frame per second. The use of a cascade of increasingly precise human annotations ensures a label accuracy above 95% for every class and tight bounding boxes. Finally, we train and evaluate well-known deep network architectures and report baseline figures for per-frame classification and localization. We also demonstrate how the temporal contiguity of video can potentially be used to improve such inferences. The data set can be found at https://research.google.com/youtube-bb. We hope the availability of such large curated corpus will spur new advances in video object detection and tracking.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, methodology): “We also consider video datasets, IMAGENET-VID and YOUTUBE-BB; we use both the accuracy metric and the pm-10 metric (suffix -”
Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “[67] Real, E.”
Do Image Classifiers Generalize Across Time? (2019, methodology): “As a cornerstone of our investigation, we introduce two test sets for evaluating model robustness: ImageNet-Vid-Robust and YTBB-Robust, carefully curated from the ImageNet-Vid and Youtube-BB datasets, respectively (Russakovsky et al., 2015; Real et al., 2017).”
Using Videos to Evaluate Image Model Robustness (2019, methodology): “We leverage a large scale dataset of videos, YouTube-BoundingBoxes(YT-BB) (Real et al. (2017)), to evaluate natural robustness.”
Generate to Adapt: Aligning Domains Using Generative Adversarial Networks (2017, methodology): “which are drawn from Microsoft COCO [14](validation set) and Youtube Bounding Box dataset [28](test set).”
Self-ensembling for visual domain adaptation (2017, methodology): “…challenge is a 12-class domain adaptation problem consisting of three datasets: a training set consisting of 3D renderings of sketchup models, and validation and test sets consisting of real images (see Figure 1) drawn from the COCO [16] and YouTube BoundingBoxes [20] datasets respectively.”
A Review of Single-Source Deep Unsupervised Visual Domain Adaptation (2020, methodology): “The synthetic images are collected from 3D CAD models while the real-world images are from COCO [171] and YouTube Bounding Boxes [172].”
VisDA: A Synthetic-to-Real Benchmark for Visual Domain Adaptation (2018, methodology): “The top group shows synthetically rendered images (source domain), the middle group shows objects cropped from COCO dataset [9] using their bounding boxes (validation target domain), and the bottom group shows similarly cropped images from YouTube-BB dataset [13] (test target domain).”
Domain2Vec: Domain Embedding for Unsupervised Domain Adaptation (2020, methodology): “Instead, in our work, we demo our model on the following four transfer setting: DomainNet [16]→VisDA [53], DomainNet→Youtube BBox [72], DomainNet→PASCAL [64], DomainNet→COCO.”
Graph Attention Tracking (2020, unknown): “We adopt COCO [19], ImageNet DET [25], ImageNet VID [25], YouTube-BB [23] and GOT-10k [14] as the training set for experiments on OTB100 [31] and UAV123 [21].”
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking (2018, methodology): “TrackingNet [41] is a subset of the video object detection benchmark YT-BB [43] by selecting 30K videos, each of which is annotated by a tracker.”
The Seventh Visual Object Tracking VOT2019 Challenge Results (2019, background): “Mueller et al. [74] recently re-annotated selected sequences from YouTube bounding boxes [78] to consider tracking in the wild.”
Siamese Box Adaptive Network for Visual Tracking (2020, methodology): “• We design a Siamese box adaptive network, which can perform end-to-end offline training with deep convolutional neural networks [12] on well-annotated datasets [34, 30, 25, 15, 9].”
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild (2018, methodology): “Datasets that are mostly used for training trackers include ALOV++ [21], NUS_PRO [17], ImageNet-VID [23], YouTube-BB [26], LaSOT [20] and TrackingNet [19].”