Progressive learning: A deep learning framework for continual learning
By Haytham M. Fayek, Lawrence J. Cavedon, Hong Ren Wu. 2020
85 citations recorded by the source corpus.
Continual learning is the ability of a learning system to solve new tasks by utilizing previously acquired knowledge from learning and performing prior tasks without having significant adverse effects on the acquired prior knowledge. Continual learning is key to advancing machine learning and artificial intelligence. Progressive learning is a deep learning framework for continual learning that comprises three procedures: curriculum, progression, and pruning. The curriculum procedure is used to actively select a task to learn from a set of candidate tasks. The progression procedure is used to grow the capacity of the model by adding new parameters that leverage parameters learned in prior tasks, while learning from data available for the new task at hand, without being susceptible to catastrophic forgetting. The pruning procedure is used to counteract the growth in the number of parameters as further tasks are learned, as well as to mitigate negative forward transfer, in which prior knowledge unrelated to the task at hand may interfere and worsen performance. Progressive learning is evaluated on a number of supervised classification tasks in the image recognition and speech recognition domains to demonstrate its advantages compared with baseline methods. It is shown that, when tasks are related, progressive learning leads to faster learning that converges to better generalization performance using a smaller number of dedicated parameters.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Radiology, Nuclear Medicine and Imaging
Citations in the corpus
- Genetic dissection of mutual interference between two consecutive learning tasks in Drosophila (2022, background): “Continual learning is a natural ability of animals ranging from invertebrates to vertebrates but a great challenge for artificial intelligence (Fayek et al., 2020; Kudithipudi et al., 2022; Parisi et al., 2019; Wang et al., 2023).”