Incremental Few-Shot Instance Segmentation

By Dan Andrei Ganea, B. Boom, Ronald Poppe. 2021

92 citations recorded by the source corpus.

Few-shot instance segmentation methods are promising when labeled training data for novel classes is scarce. However, current approaches do not facilitate flexible addition of novel classes. They also require that examples of each class are provided at train and test time, which is memory intensive. In this paper, we address these limitations by presenting the first incremental approach to few-shot instance segmentation: iMTFA. We learn discriminative embeddings for object instances that are merged into class representatives. Storing embedding vectors rather than images effectively solves the memory overhead problem. We match these class embeddings at the RoI-level using cosine similarity. This allows us to add new classes without the need for further training or access to previous training data. In a series of experiments, we consistently outperform the current state-of-the-art. Moreover, the reduced memory requirements allow us to evaluate, for the first time, few-shot instance segmentation performance on all classes in COCO jointly1.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Ziwei Liu
5. Hang Su
6. Qiwen Wang
7. Jun Zhu
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Han-Jia Ye
3. Ziwei Liu
4. De-chuan Zhan
5. Bernt Schiele
6. Jun Zhu
7. Trevor Darrell
8. Dacheng Tao

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “In addition to the regular CSS, other relevant settings include unsupervised domain adaptation for CSS [394], where a pre-trained model adapts to the unannotated data of the target domain without using the annotated data of the source domain; incremental few-shot semantic segmentation [379], which p”
• Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation (2022, methodology): “Regularization-based methods [8, 11, 25] protect old knowledge from being covered by imposing constraints on new tasks.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “[117] applied the memory basedmethod on fewshot image segmentation.”
• A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, methodology): “Method Venue Backbone Type Tricks COCO-20i Available Code PT FT DA AT SS MTL 1-shot 5-shot iMTFA [264] CVPR’21 R-CNN Metric ✓ ✓ ✓ 20.”
• Class-Incremental Learning: A Survey (2023, background): “Recent works in CSS also involve incremental learning in more challenging scenarios, e.g. , learning with few-shot annotations [283], [284], different domains [285], [286], continual instance segmentation [287], [288] and weakly semantic segmentation [289].”
• Prototype-based Incremental Few-Shot Segmentation (2020, background): “Concurrently to us, [14] proposed the incremental few-shot instance segmentation setting.”
• A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “For more challenging incremental few-shot instance segmentation, Ganea et al. (2021) introduced Model agnostic meth-ods and proposed the first approach to solving this problem: iMTFA.”
• Few-Shot Incremental Learning for Label-to-Image Translation (2022, methodology): “To alleviate the dependency on the amount of data for new tasks, advanced methods are proposed to solve fewshot incremental learning problems [11, 21, 29, 61].”