Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning

By Zhuowei Li, Long Zhao, Zizhao Zhang, Diya Liu, Ting Liu, Dimitris N. Metaxas. 2023

43 citations recorded by the source corpus.

In the context of continual learning, prototypes—as representative class embeddings—offer advantages in memory conservation and the mitigation of catastrophic forgetting. However, challenges related to semantic drift and prototype interference persist. In this study, we introduce the Contrastive Prototypical Prompt (CPP) approach. Through task-specific prompt-tuning, underpinned by a contrastive learning objective, we effectively address both aforementioned challenges. Our evaluations on four challenging class-incremental benchmarks reveal that CPP achieves a significant 4% to 6% improvement over state-of-the-art methods. Importantly, CPP operates without a rehearsal buffer and narrows the performance divergence between continual and offline joint-learning, suggesting an innovative scheme for Transformer-based continual learning systems1.

Subfields: Artificial Intelligence, Control and Systems Engineering, Media Technology

Citations in the corpus

• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “They proposed Variational Auto-Regressive Gaussian Processes, which uses sparse inducing point approximation to better approximate the Gaussian posterior, resulting in a lower bound objective for regularization.”