Recent Advances of Continual Learning in Computer Vision: An Overview

By Haoxuan Qu, Hossein Rahmani, Li Xu, Bryan M. Williams, Jun Liu. 2021

114 citations recorded by the source corpus.

In contrast to batch learning where all training data is available at once, continual learning represents a family of methods that accumulate knowledge and learn continuously with data available in sequential order. Similar to the human learning process with the ability of learning, fusing and accumulating new knowledge acquired at different time steps, continual learning is considered to have high practical significance. Hence, continual learning has been studied in various artificial intelligence tasks. In this paper, we present a comprehensive review of the recent progress of continual learning in computer vision. In particular, the works are grouped by their representative techniques, including regularisation, knowledge distillation, memory, generative replay, parameter isolation and a combination of the above techniques. For each category of these techniques, both its characteristics and applications in computer vision are presented. At the end of this overview, several subareas, where continuous knowledge accumulation is potentially helpful while continual learning has not been well studied, are discussed.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “In contrast, the latest surveystypicallycollateonlyalocalaspectofcontinuallearning, withrespecttoitsbiologicalunderpinnings[1],[3],[9],[10], specializedsettingsforvisualclassiﬁcation[11],[12],[13],[14], andspeciﬁcextensionsforNLP[15],[16]orRL[17].”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “…Ramesh & Chaudhari, 2022; Douillard et al., 2022), or combinations thereof (Yan et al., 2021; Xie et al., 2022; Wang et al., 2022b; Frascaroli et al., 2023); for surveys, see, e.g., Parisi et al. (2019); De Lange et al. (2022); Qu et al. (2021); Shaheen et al. (2022); van de Ven et al. (2022).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Other surveys have focused on the use of lifelong learning for specific applications, such as vision (Qu et al., 2021), language (Biesialska et al., 2020), and robotics (Lesort et al., 2019).”
How catastrophic can catastrophic forgetting be in linear regression? (2022, background): “13) [55] Haoxuan Qu, Hossein Rahmani, Li Xu, Bryan Williams, and Jun Liu.”
Experts Collaboration Learning for Continual Multi-Modal Reasoning (2023, background): “, performing continual learning [14], [15], [16] for better serving the users.”