Learning Equi-Angular Representations for Online Continual Learning

By Minhyuk Seo, Hyun-woo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo J. Kim, Jonghyun Choi. 2024

25 citations recorded by the source corpus.

Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we propose an efficient online continual learning method using the neural collapse phenomenon. In particular, we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K, we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaus-sian scheduled continuous (i.e., boundary-free) data se-tups. Code is available at https://github.com/yonseivnl/ear/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

Citations in the corpus

• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “In image semantic segmentation, Wu et al. [211] proposed to store feature statistics of previous images to help train the image generator.”