The 2017 DAVIS Challenge on Video Object Segmentation

By Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alexander Sorkine-Hornung, Luc Van Gool. 2017

1,603 citations recorded by the source corpus.

We present the 2017 DAVIS Challenge on Video Object Segmentation, a public dataset, benchmark, and competition specifically designed for the task of video object segmentation. Following the footsteps of other successful initiatives, such as ILSVRC and PASCAL VOC, which established the avenue of research in the fields of scene classification and semantic segmentation, the DAVIS Challenge comprises a dataset, an evaluation methodology, and a public competition with a dedicated workshop co-located with CVPR 2017. The DAVIS Challenge follows up on the recent publication of DAVIS (Densely-Annotated VIdeo Segmentation), which has fostered the development of several novel state-of-the-art video object segmentation techniques. In this paper we describe the scope of the benchmark, highlight the main characteristics of the dataset, define the evaluation metrics of the competition, and present a detailed analysis of the results of the participants to the challenge.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Emerging Properties in Self-Supervised Vision Transformers (2021, neutral): “5, we evaluate the output patch tokens on the DAVIS-2017 video instance segmentation benchmark [52].”
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations (2022, methodology): “2 Metrics and Evaluation We report our results in this benchmark using two measures as proposed in [32]: (1) Jaccard Index (J ) Given the ground-truth G and predicted mask P , the Jaccard Index is defined as J = (∣P ∩G∣)/∣P ∪G∣).”
Deep Extreme Cut: From Extreme Points to Object Segmentation (2017, methodology): “Our method is extensively validated in ﬁve publicly available databases: PASCAL [8], COCO [21], DAVIS-2016 [27], DAVIS-2017 [31], and Grabcut [32], for various experimental setups that show its applicability and generalization capabilities.”
Tracking Emerges by Colorizing Videos (2018, methodology): “The videos in DAVIS 2017 are challenging and consist of multiple objects that undergo signiﬁcant deformation, occlusion, and scale change with cluttered backgrounds.”
YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark (2018, background): “In addition, their video content is relatively simple [15,20] was published and has become the benchmark dataset in this area.”
Video Instance Segmentation (2019, background): “Existing video segmentation datasets either do not have exhaustive labeling [21, 31], or do not have the object identities [7].”
Lucid Data Dreaming for Object Tracking (2017, methodology): “For multiple object tracking we experiment on DAVIS 2017 [16].”
Learning Correspondence From the Cycle-Consistency of Time (2019, methodology): “Table 1: Evaluation on instance mask propagation on DAVIS2017 [48].”
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors (2023, methodology): “We collect those input images from the Internet, DAVIS [54], and JourneyDB [53].”
Kubric: A scalable dataset generator (2022, methodology): “One of the closest problems in the literature is segment tracking, using datasets like DAVIS [70].”
Survey on semantic segmentation using deep learning techniques (2019, background): “objects, ADE20K [194] containing both indoor and outdoor images with large variations, and DAVIS [195] dataset containing densely annotated videos with pixel accurate ground truth.”
Text2LIVE: Text-Driven Layered Image and Video Editing (2022, methodology): “The video set contains seven videos from DAVIS dataset [34].”
Personalize Segment Anything Model with One Shot (2023, methodology): “We select the popular DAVIS 2017 [40] dataset for evaluation, and adopt the official J and F scores as metrics.”
Video Object Segmentation without Temporal Information (2017, methodology): “DAVIS Validation : Per-sequence results of mean region similarity ( J ) and contour accuracy ( F ). we also add the most recent mean between J and F , as used in the 2017 DAVIS Challenge [51].”
TokenFlow: Consistent Diffusion Features for Consistent Video Editing (2023, methodology): “We evaluate our method on DAVIS videos [33] and on Internet videos depicting animals, food, humans, and various objects in motion.”
SiT: Self-supervised vIsion Transformer (2021, methodology): “In this experiment, we demonstrate the generalisation of the frozen learnt features by SiT, pretrained on ImageNet1K, on the DAVIS-2017 video instance segmentation benchmark [80].”
Segment Everything Everywhere All at Once (2023, methodology): “As shown in Table 3, without any observation of DAVIS/VOS dataset [71, 72], our approach is able to achieve close performance in a zero-shot manner with a fully supervised method on DAVIS17 dataset [72].”
The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation (2019, methodology): “The 2017 DAVIS Challenge on Video Object Segmentation [2] presented an extension of the dataset: 150 sequences (10474 annotated frames) instead of 50 (3455 frames), more than one annotated object per sequence (384 objects instead of 50), and more challenging scenarios such as motion blur,…”
Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation (2021, methodology): “For comparing the speed, we compute the multi-object FPS that is the total number of output frames divided by the total processing time for the entire DAVIS 2017 [65] validation set.”