Learn++: an incremental learning algorithm for supervised neural networks

By Robi Polikar, L. Upda, S. Upda, Vasant G Honavar. 2001

946 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

In Meta & Continual Learning

Citations in the corpus

Model-Reuse Attacks on Deep Learning Systems (2018, background): “One may suggest using incremental learning [15, 48], which retrains an existing model to accommodate new data, or open-set learning, which extends a given model to new classes during inference [4, 51].”
iCaRL: Incremental Classifier and Representation Learning (2016, background): “[28, 30] introduced an ensemble based approach that can handle an increasing number of classes but needs training data for all classes to occur repeatedly.”
Measuring Catastrophic Forgetting in Neural Networks (2017, methodology): “For the explicit methods, such as Learn++ and TradaBoost, this prevents forgetting because an entirely new sub-network is trained for a new batch (Polikar et al. 2001; Dai et al. 2007).”
Memory Efficient Experience Replay for Streaming Learning (2018, methodology): “Ensembling methods train multiple models and combine their outputs to make predictions [17], [21], [62], [65], [70].”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “One of the most popular approaches to relieve forgetting is by imposing a regularization loss within the optimization procedure [7, 11, 13, 16, 19, 25, 26, 31, 34, 35, 40, 41, 57], where the network’s parameters which are important to the past tasks are penalized when updating.”
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches (2022, background): “Incremental Learning: Incremental Learning (IL) [38, 25] is a machine learning paradigm where a model adapts itself to learn new tasks sequentially while retaining the previously learnt knowledge.”
Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, background): “Regularization approaches in order to alleviate catastrophic forgetting add an auxiliary term that penalizes changes in the weights when the model is trained on a new task [14,26,29,33,38,48,54,60].”
Incremental Learning in Online Scenario (2020, methodology): “Other techniques [19, 17, 13] use ensemble of weak classifiers and nearest neighbor classifier.”
Knowledge Distillation for Incremental Learning in Semantic Segmentation (2019, background): “Catastrophic forgetting has been faced even before the rise of neural networks popularity [26, 27, 28] and more recently has been rediscovered and tackled in different ways.”
Incremental Learning Techniques for Semantic Segmentation (2019, background): “It has been addressed even before the rise of neural networks popularity [6, 25, 33], but more recently it has been rediscovered and tackled in different ways.”
Large Scale Incremental Learning (2019, background): “Incremental learning has been a long standing problem in machine learning [3, 17, 16, 12].”
Incremental Learning of Object Detectors without Catastrophic Forgetting (2017, background): “The problem of incremental learning has a long history in machine learning and artificial intelligence [6,29,36,37].”
Incremental Classifier Learning with Generative Adversarial Networks (2018, background): “Related Work Incremental learning has been a long standing problem in machine learning [3, 20, 17, 12].”
A Strategy for an Uncompromising Incremental Learner (2017, methodology): “The Learn++ is an ensemble based approach for incremental learning [23] [20].”
RODEO: Replay for Online Object Detection (2020, neutral): “work sparsely updates weights to mitigate interference [8], 3) ensembling multiple classifiers together [9, 11, 43, 47, 53], and 4) rehearsal/replay models that store a subset of previous training inputs (or generate previous inputs) to mix with new examples when updating the network [4, 16, 17, 21,”
Balancing Between Forgetting and Acquisition in Incremental Subpopulation Learning (2022, methodology): “Other kinds of NEB methods learn a generative model (GAN) [30,34,4,34] to generate the old images for retraining or dynamically extend the models [23,22,38,5,37].”
Incremental on-line learning: A review and comparison of state of the art algorithms (2018, background): “Furthermore, the ORF benefits more from the sub-linear tree complexity because it generates a few deep trees instead of the numerous, shallow ones as done by LPPCART.”