Class-Incremental Learning for Action Recognition in Videos

By Jaeyoo Park, Minsoo Kang, Bohyung Han. 2021

69 citations recorded by the source corpus.

We tackle catastrophic forgetting problem in the context of class-incremental learning for video recognition, which has not been explored actively despite the popularity of continual learning. Our framework addresses this challenging task by introducing time-channel importance maps and exploiting the importance maps for learning the representations of incoming examples via knowledge distillation. We also incorporate a regularization scheme in our objective function, which encourages individual features obtained from different time steps in a video to be uncorrelated and eventually improves accuracy by alleviating catastrophic forgetting. We evaluate the proposed approach on brand-new splits of class-incremental action recognition benchmarks constructed upon the UCF101, HMDB51, and Something-Something V2 datasets, and demonstrate the effectiveness of our algorithm in comparison to the existing continual learning methods that are originally designed for image data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “There are also many benchmarks for video classification [275], [324], [363], [418], varying in size and purpose.”
Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation (2022, methodology): “Different from the previous works including the closely related method [35], our approach estimates the importance of each feature map for knowledge distillation and provides the theoretical foundation for the adaptive weighting of feature maps.”
vCLIMB: A Novel Video Class Incremental Learning Benchmark (2022, methodology): “UCF101-50 [26] 51 classes 5/10/25 – – – 10/5/2 – ✗ HMDB51-25 [26] 26 classes 5/25 – – – 5/1 – ✗ Something-Something V2-90 [26] 84 classes 9/18 – – – 10/5 – ✗”
Class-Incremental Learning: A Survey (2023, background): “The shade color in the last column denotes the subcategory, which is consistent with Figure 3.”
Coherent Temporal Synthesis for Incremental Action Segmentation (2024, unknown): “Action recognition [23], as the hallmark task of video understanding, was the initial focus for incremental learning for videos [3, 36, 37, 50].”
Online Continual Learning For Interactive Instruction Following Agents (2024, background): “Recently, continual learning has also been actively investigated in more challenging task setups such as video domains (Zhao et al., 2021b; Villa et al., 2022; Park et al., 2021).”
Deep Class-Incremental Learning: A Survey (2023, background): “, semantic segmentation [164], [165], [166], [167], [174], object detection [175], [176], [177], [178], [179], [180], [181], video understanding [95], [133], medical surgery [182], [183], language model [184], [185], [186], [187], language-vision model [188], [189], etc.”