vCLIMB: A Novel Video Class Incremental Learning Benchmark
By Andrés Villa, Kumail Alhamoud, Juan Le'on Alc'azar, Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem. 2022
46 citations recorded by the source corpus.
Continual learning (CL) is under-explored in the video domain. The few existing works contain splits with imbalanced class distributions over the tasks, or study the problem in unsuitable datasets. We introduce vCLIMB, a novel video continual learning benchmark. vCLIMB is a standardized test-bed to analyze catastrophic forgetting of deep models in video continual learning. In contrast to previous work, we focus on class incremental continual learning with models trained on a sequence of disjoint tasks, and distribute the number of classes uniformly across the tasks. We perform in-depth evaluations of existing CL methods in vCLIMB, and observe two unique challenges in video data. The selection of instances to store in episodic memory is performed at the frame level. Second, untrimmed training data influences the effectiveness of frame sampling strategies. We address these two challenges by proposing a temporal consistency regularization that can be applied on top of memory-based continual learning methods. Our approach significantly improves the baseline, by up to 24% on the untrimmed continual learning task. The code of our benchmark can be found at: https://vclimb.netlify.app/.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Evaluation & Benchmarks
In Meta & Continual Learning
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “There are also many benchmarks for video classification [275], [324], [363], [418], varying in size and purpose.”
- Class-Incremental Learning: A Survey (2023, unknown): “…segmentation [193], [238], [239], [240], [241], object detection [137], [242], [243], [244], [245], [246], [247], video understanding [91], [161], medical surgery [248], [249], language model [250], [251], [252], [253], language-vision model [217], [254], etc. Specifically, object…”
- Coherent Temporal Synthesis for Incremental Action Segmentation (2024, unknown): “Storing frame-exemplars for videos requires a balance between dense temporal sampling and memory diversity [3, 50].”
- Deep Class-Incremental Learning: A Survey (2023, methodology): “, semantic segmentation [164], [165], [166], [167], [174], object detection [175], [176], [177], [178], [179], [180], [181], video understanding [95], [133], medical surgery [182], [183], language model [184], [185], [186], [187], language-vision model [188], [189], etc.”
- Online Continual Learning For Interactive Instruction Following Agents (2024, background): “Recently, continual learning has also been actively investigated in more challenging task setups such as video domains (Zhao et al., 2021b; Villa et al., 2022; Park et al., 2021).”