LVIS: A Dataset for Large Vocabulary Instance Segmentation

By Agrim Gupta, Piotr Doll'ar, Ross B. Girshick. 2019

1,821 citations recorded by the source corpus.

Progress on object detection is enabled by datasets that focus the research community’s attention on open challenges. This process led us from simple images to complex scenes and from bounding boxes to segmentation masks. In this work, we introduce LVIS (pronounced ‘el-vis’): a new dataset for Large Vocabulary Instance Segmentation. We plan to collect 2.2 million high-quality instance segmentation masks for over 1000 entry-level object categories in 164k images. Due to the Zipfian distribution of categories in natural images, LVIS naturally has a long tail of categories with few training samples. Given that state-of-the-art deep learning methods for object detection perform poorly in the low-sample regime, we believe that our dataset poses an important and exciting new scientific challenge. LVIS is available at http://www.lvisdataset.org.

Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

Momentum Contrast for Unsupervised Visual Representation Learning (2019, background): “5 instance segmentation [27]: this task has ∼1000 long-tailed distributed categories.”
Segment Anything (2023, methodology): “ADE20K[117] BBBC038v1[12] Cityscapes[25] DOORS[80] DRAM[24] EgoHOS[113] GTEA[34,63] Hypersim[86] IBD[17] iShape[111] LVIS[44] NDD20[100] NDISPark[22,23] OVIS[81] PPDLS[74] Plittersdorf[46] STREETS[91] TimberSeg[38] TrashCan[52] VISOR[28,27] WoodScape[112] PIDRay[104] ZeroWaste-f[6] Figure 8: Samples”
Florence: A New Foundation Model for Computer Vision (2021, methodology): “We merge several wellknown object detection datasets, including COCO (Lin et al., 2015), LVIS (Gupta et al., 2019), OpenImages (Krasin et al., 2016), Object365 (Shao et al., 2019).”
A Simple Framework for Open-Vocabulary Segmentation and Detection (2023, methodology): “In addition, we evaluate both segmentation and detection performance on LVIS [14].”
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection (2023, background): “LVIS [14] is a dataset for long-tail objects.”
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models (2022, background): “2M / - 940 X X ELEVATER (Ours) 20 638K / 193K 1151♦ WordNet, Wiki, GPT-3 X X X OD LVIS [21] 1 120k / 40K 1723 WordNet X ELEVATER (Ours) 35 132K / 20K 314♦ WordNet, Wiki, GPT-3 X X X Table 1: Comparison of dataset statistics and evaluation settings.”
RegionCLIP: Region-based Language-Image Pretraining (2021, methodology): “, Conceptual Caption [45]) and mainly evaluate our method on the benchmarks of open-vocabulary object detection (COCO [32] and LVIS [19] datasets).”
Grounded Language-Image Pre-training (2021, background): “Adding image-text data further widens the gap on LVIS AP r (20.8 versus 15.0).”
SimpleClick: Interactive Image Segmentation with Simple Vision Transformers (2022, methodology): “• COCO [29]+ LVIS [19] (C+L): COCO contains 118K training images (1.2M instances); LVIS shares the same images with COCO but has much higher segmentation quality.”
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations (2022, methodology): “1, 3 [17] Agrim Gupta, Piotr Dollar, and Ross Girshick.”
Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, methodology): “COCO also exhaustively annotates all masks of objects that belong to its 80 categories while LVIS is annotated so as to maintain a similar number of masks across categories.”
FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “56 Ours-hrnet18s-S2 COCO [26]+LVIS [13] 1.”
Exploring Plain Vision Transformer Backbones for Object Detection (2022, methodology): “We follow the same model and training details as used for the COCO systemlevel comparison plus two common LVIS practices: we use the federated loss from [57] and sample images with repeat factor sampling [22].”
Learning Open-World Object Proposals without Learning to Classify (2021, background): “Finally, we study the 4) impact on long-tail object detection of OLN on LVIS dataset.”
Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, methodology): “[5], LVIS has the highest annotation quality among the aforementioned datasets.”
Occluded Video Instance Segmentation: A Benchmark (2021, background): “The categories are carefully chosenmainly for three motivations: (1) most of them are animals, with which object occlusions extensively happen, (2) they are commonly seen in our life, (3) these categories have a high overlap with popular large-scale image instance segmentation datasets (Gupta et al.”
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, methodology): “One common method is Repeat Factor Sampling (RFS) from [21], with t = 0.”