Effects of Auxiliary Knowledge on Continual Learning
By Giovanni Bellitto, Matteo Pennisi, Simone Palazzo, Lorenzo Bonicelli, Matteo Boschini, Simone Calderara, Concetto Spampinato. 2022
9 citations recorded by the source corpus.
In Continual Learning (CL), a neural network is trained on a stream of data whose distribution changes over time. In this context, the main problem is how to learn new information without forgetting old knowledge (i.e., Catastrophic Forgetting). Most existing CL approaches focus on finding solutions to preserve acquired knowledge, so working on the "past" of the model. However, we argue that as the model has to continually learn new tasks, it is also important to put focus on the "present" knowledge that could improve following tasks learning. In this paper we propose a new, simple, CL algorithm that focuses on solving the current task in a way that might facilitate the learning of the next ones. More specifically, our approach combines the main data stream with a secondary, diverse and uncorrelated stream, from which the network can draw auxiliary knowledge. This helps the model from different perspectives, since auxiliary data may contain useful features for the current and the next tasks and incoming task classes can be mapped onto auxiliary classes. Furthermore, the addition of data to the current task is implicitly making the classifier more robust as we are forcing the extraction of more discriminative features. Our method can outperform existing state-of-the-art models on the most common CL Image Classification benchmarks.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Matteo Boschini
- Lorenzo Bonicelli
- Simone Calderara
- Concetto Spampinato
- Angelo Porrello
- Simone Palazzo
- Giovanni Bellitto
- Matteo Pennisi
In Computer Vision and Pattern Recognition
- Matteo Boschini
- Simone Calderara
- Lorenzo Bonicelli
- Angelo Porrello
- Concetto Spampinato
- Giovanni Bellitto
- Matteo Pennisi
- Simone Palazzo
In Radiology, Nuclear Medicine and Imaging
- Lorenzo Bonicelli
- Simone Calderara
- Matteo Boschini
- Angelo Porrello
- Gyuhak Kim
- Zixuan Ke
- Bahram Zonooz
- Elahe Arani
Citations in the corpus
- On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, methodology): “To assess our proposal, we perform a suite of experiments through the Mammoth framework [12, 17, 9, 6, 27, 13, 14, 50], an open-source codebase introduced in [15] for testing CL algorithms.”
- Transfer without Forgetting (2022, background): “An alternative for preserving the source knowledge is to assume that pretraining data is available and can be treated as an auxiliary data stream [5].”