Coherent Temporal Synthesis for Incremental Action Segmentation

By Guodong Ding, Hans Golong, Angela Yao. 2024

8 citations recorded by the source corpus.

Data replay is a successful incremental learning technique for images. It prevents catastrophic forgetting by keeping a reservoir of previous data, original or synthe-sized, to ensure the model retains past knowledge while adapting to novel concepts. However, its application in the video domain is rudimentary, as it simply stores frame ex-emplars for action recognition. This paper presents the first exploration of video data replay techniques for incremen-tal action segmentation, focusing on action temporal modeling. We propose a Temporally Coherent Action (TCA) model, which represents actions using a generative model instead of storing individual frames. The integration of a conditioning variable that captures temporal coherence al-lows our model to understand the evolution of action features over time. Therefore, action segments generated by TCA for replay are diverse and temporally coherent. In a 10-task incremental setup on the Breakfast dataset, our approach achieves significant increases in accuracy for up to 22% compared to the baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

Citations in the corpus

• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Based on this, they proposed a generative replay based continual learning method for NeRFs and further incorporated their method with an instant neural graphics primitives [139] architecture.”