Learning to Segment the Tail

By Xinting Hu, Yi Jiang, Kaihua Tang, Jingyuan Chen, Chunyan Miao, Hanwang Zhang. 2020

93 citations recorded by the source corpus.

Real-world visual recognition requires handling the extreme sample imbalance in large-scale long-tailed data. We propose a “divide&conquer” strategy for the challenging LVIS task: divide the whole data into balanced parts and then apply incremental learning to conquer each one. This derives a novel learning paradigm: class-incremental few-shot learning, which is especially effective for the challenge evolving over time: 1) the class imbalance among the old class knowledge review and 2) the few-shot data in new-class learning. We call our approach Learning to Segment the Tail (LST). In particular, we design an instance-level balanced replay scheme, which is a memory-efficient approximation to balance the instance-level samples from the old-class images. We also propose to use a meta-module for new-class learning, where the module parameters are shared across incremental phases, gaining the learning-to-learn knowledge incrementally, from the data-rich head to the data-poor tail. We empirically show that: at the expense of a little sacrifice of head-class forgetting, we can gain a significant 8.3% AP improvement for the tail classes with less than 10 instances, achieving an overall 2.0% AP boost for the whole 1,230 classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Reviving Iterative Training with Mask Guidance for Interactive Segmentation (2021, background): “These datasets stimulated the development of state-of-the-art instance segmentation algorithms [42, 43, 44, 45], providing a large variety of labeled classes with a sufficient number of examples.”
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (2020, background): “Other more complicated learning methods ( e.g ., meta-learning [63, 29, 32], causal inference [58], Bayesian methods [34], etc .) are also used to deal with long-tail data.”
Deep Long-Tailed Learning: A Survey (2021, background): “Class-Incremental Long-Tailed Learning: In real-world applications, long-tailed data may come in a continual and classincremental manner [98], [168], [169].”
Seesaw Loss for Long-Tailed Instance Segmentation (2020, background): “Other common approches [15, 19, 34] re-balance the distribution of the instance numbers in each class, e.g ., repeat factor sampling [14] and class-balanced sampling [34], both are based on the sample numbers of classes.”
Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect (2020, unknown): “It can be either achieved by re-sampling [25, 26, 15, 16, 27] or re-weighting [13, 14, 12, 17].”
Boosting Long-tailed Object Detection via Step-wise Learning on Smooth-tail Data (2023, methodology): “[10] proposes LST which is a "divide & conquer" strategy that leverages class-incremental few-shot learning to solve the long-tailed distribution problem.”
Online Continual Learning for Embedded Devices (2022, background): “…al., 2017; Castro et al., 2018; Wu et al., 2019), video streams (Hayes et al., 2020; Hayes & Kanan, 2020; Pellegrini et al., 2020; 2021), imbalanced data streams (Aggarwal et al., 2021; Belouadah et al., 2020; Hu et al., 2020), and low-shot learning (Ayub & Wagner, 2020; 2021; Tao et al., 2020b;a).”