The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation

By Tao Wang, Yu Li, Bingyi Kang, Junnan Li, J. Liew, Sheng Tang, Steven C. H. Hoi, Jiashi Feng. 2020

207 citations recorded by the source corpus.

Most existing object instance detection and segmentation models only work well on fairly balanced benchmarks where per-category training sample numbers are comparable, such as COCO. They tend to suffer performance drop on realistic datasets that are usually long-tailed. This work aims to study and address such open challenges. Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset, and unveil that a major cause is the inaccurate classification of object proposals. Based on such an observation, we first consider various techniques for improving long-tail classification performance which indeed enhance instance segmentation results. We then propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach. Without bells and whistles, it significantly boosts the performance of instance segmentation for tail classes on the recent LVIS dataset and our sampled COCO-LT dataset. Our analysis provides useful insights for solving long-tail instance detection and segmentation problems, and the straightforward \emph{SimCal} method can serve as a simple but strong baseline. With the method we have won the 2019 LVIS challenge. Codes and models are available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chen Change Loy
2. Ziwei Liu
3. Dahua Lin
4. Kai Chen
5. Anelia Angelova
6. Jiangmiao Pang
7. Jian‐Yuan Sun
8. Weicheng Kuo

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Chen Change Loy
3. Dacheng Tao
4. Xiaogang Wang
5. Ming-Hsuan Yang
6. Jitendra Malik
7. Trevor Darrell
8. Alan L. Yuille

In Radiology, Nuclear Medicine and Imaging

1. Xuming He
2. Shipeng Yan
3. Ziwei Liu
4. Tsung-Yi Lin
5. Kai Chen
6. Chen Change Loy
7. Songyang Zhang
8. Dahua Lin

Citations in the corpus

• Learning Open-World Object Proposals without Learning to Classify (2021, background): “Large vocabulary detection has gained much attention lately in research community [22, 24, 52, 65] because the problem captures the Zipf’s law of visual world, i.”
• Seesaw Loss for Long-Tailed Instance Segmentation (2020, methodology): “There are also attempts to modify the classifier to improve the performance on tail classes, e.g ., using different classifiers for different groups of classes [26], or use two classifiers trained with different data samplers [53].”
• Boosting Long-tailed Object Detection via Step-wise Learning on Smooth-tail Data (2023, methodology): “[31] proposes a calibration framework to alleviate classification head bias with a bi-level class balanced sampling approach at instance-level.”
• Distribution Alignment: A Unified Framework for Long-tail Visual Recognition (2021, methodology): “The similar idea is adopted in [38, 39, 23], which develop effective strategies for long-tail object detection tasks.”