Class-Incremental Instance Segmentation via Multi-Teacher Networks

By Yanan Gu, Cheng Deng, Kun-Juan Wei. 2021

32 citations recorded by the source corpus.

Although deep neural networks have achieved amazing results on instance segmentation, they are still ill-equipped when they are required to learn new tasks incrementally. Concretely, they suffer from “catastrophic forgetting”, an abrupt degradation of performance on old classes with the initial training data missing. Moreover, they are subjected to a negative transfer problem on new classes, which renders the model unable to update its knowledge while preserving the previous knowledge. To address these problems, we propose an incremental instance segmentation method that consists of three networks: Former Teacher Network (FTN), Current Student Network (CSN) and Current Teacher Network (CTN). Specifically, FTN supervises CSN to preserve the previous knowledge, and CTN supervises CSN to adapt to new classes. The supervision of two teacher networks is achieved by a distillation loss function for instances, bounding boxes, and classes. In addition, we adjust the supervision weights of different teacher networks to balance between the knowledge preservation for former classes and the adaption to new classes. Extensive experimental results on PASCAL 2012 SBD and COCO datasets show the effectiveness of the proposed method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Qiwen Wang
6. Jun Zhu
7. Ziwei Liu
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Ziwei Liu
5. Luc Van Gool
6. Dacheng Tao
7. Hang Su
8. Da-Wei Zhou

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “In addition to the regular CSS, other relevant settings include unsupervised domain adaptation for CSS [394], where a pre-trained model adapts to the unannotated data of the target domain without using the annotated data of the source domain; incremental few-shot semantic segmentation [379], which p”
• Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation (2022, background): “Recently, incremental learning has been extensively studied for image segmentation task [13, 33, 45].”
• Not Just Selection, but Exploration: Online Class-Incremental Continual Learning via Dual View Consistency (2022, background): “Continual learning [21, 32, 35, 37, 51, 52] aims to build a model that can learn sequentially and accumulate acquired knowledge over time.”
• Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning (2023, background): “In practice, by saving and replaying earlier data instances, rehearsal-based methods have consistently demonstrated versatility and resilience [4,6].”
• Class-Incremental Learning: A Survey (2023, background): “Recent works in CSS also involve incremental learning in more challenging scenarios, e.g. , learning with few-shot annotations [283], [284], different domains [285], [286], continual instance segmentation [287], [288] and weakly semantic segmentation [289].”