A Theory for Knowledge Transfer in Continual Learning

By Diana Benavides Prado, Patricia J. Riddle. 2022

23 citations recorded by the source corpus.

Continual learning of a stream of tasks is an active area in deep neural networks. The main challenge investigated has been the phenomenon of catastrophic forgetting or interference of newly acquired knowledge with knowledge from previous tasks. Recent work has investigated forward knowledge transfer to new tasks. Backward transfer for improving knowledge gained during previous tasks has received much less attention. There is in general limited understanding of how knowledge transfer could aid tasks learned continually. We present a theory for knowledge transfer in continual supervised learning, which considers both forward and backward transfer. We aim at understanding their impact for increasingly knowledgeable learners. We derive error bounds for each of these transfer mechanisms. These bounds are agnostic to specific implementations (e.g. deep neural networks). We demonstrate that, for a continual learner that observes related tasks, both forward and backward transfer can contribute to an increasing performance as more tasks are observed.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. René Vidal
2. Paris V. Giampouras
3. Liangzu Peng

Citations in the corpus

• The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “This assumption is adopted recently by Ramesh & Chaudhari (2022) and Prado & Riddle (2022) to provide insights on generalization for continual learning.”