Co-Transport for Class-Incremental Learning
By Da-Wei Zhou, Han-Jia Ye, De-chuan Zhan. 2021
100 citations recorded by the source corpus.
Traditional learning systems are trained in closed-world for a fixed number of classes, and need pre-collected datasets in advance. However, new classes often emerge in real-world applications and should be learned incrementally. For example, in electronic commerce, new types of products appear daily, and in a social media community, new topics emerge frequently. Under such circumstances, incremental models should learn several new classes at a time without forgetting. We find a strong correlation between old and new classes in incremental learning, which can be applied to relate and facilitate different learning stages mutually. As a result, we propose CO-transport for class Incremental Learning (COIL), which learns to relate across incremental tasks with the class-wise semantic relationship. In detail, co-transport has two aspects: prospective transport tries to augment the old classifier with optimal transported knowledge as fast model adaptation. Retrospective transport aims to transport new class classifiers backward as old ones to overcome forgetting. With these transports, COIL efficiently adapts to new tasks, and stably resists forgetting. Experiments on benchmark and real-world multimedia datasets validate the effectiveness of our proposed method.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Representation & Self-Supervised Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Radiology, Nuclear Medicine and Imaging
In Representation & Self-Supervised Learning
In Meta & Continual Learning
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “Label Space LwF [252], iCaRL [356], GEM [277], A-GEM [65], EEIL [53], BiC [457], WA [500], DER++ [46], ScaIL [33], S&B [218], SS-IL [11], Coil [506]”
- Anti-Retroactive Interference for Lifelong Learning (2022, methodology): “5, We compare different methods on CUB200 with 6 incremental tasks where BiC [30] and CoIL [37] are memory-based methods.”
- FOSTER: Feature Boosting and Compression for Class-Incremental Learning (2022, neutral): “Methods [1,27,32,38,43,47] retain an old model additionally and use knowledge distillation to constrain output for original tasks of the new model to be similar to that of the old one [27].”
- Forward Compatible Few-Shot Class-Incremental Learning (2022, background): “Discussion about related compatible training methods: Some other works aim to build a compact embedding space [13], which can be seen as enhancing forward compatibility implicitly.”
- PyCIL: a Python toolbox for class-incremental learning (2021, background): “Coil [9]: Builds bi-directional knowledge transfer in the incremental learning process with optimal transport.”
- Class-Incremental Learning: A Survey (2023, background): “…[33], [73], [74], [75], [76], [77], [78], [79], [80] § 3.5 Knowledge Distillation Logit Distillation [81], [82], [83], [84], [85], [86], [87], [88] Feature Distillation [89], [90], [91], [92], [93], [94], [95], [96], [97], [98] Relational Distillation [27], [99], [100], [101], [102], [103] §…”