Life-long learning based on dynamic combination model

By Boya Ren, Hongzhi Wang, Jianzhong Li, Hong Gao. 2017

47 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “Robins (1995) showed that interleaving information of new experiences with internally generated patterns of previous experiences help consolidate existing knowledge without explicitly storing training samples.”
Measuring Catastrophic Forgetting in Neural Networks (2017, methodology): “Two methods that try to alleviate the memory usage problem are Accuracy Weighted Ensem-bles and Life-long Machine Learning (Wang et al. 2003; Ren et al. 2017).”
Memory Efficient Experience Replay for Streaming Learning (2018, methodology): “Ensembling methods train multiple models and combine their outputs to make predictions [18], [22], [63], [66], [71].”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “One of the most popular approaches to relieve forgetting is by imposing a regularization loss within the optimization procedure [7, 11, 13, 16, 19, 25, 26, 31, 34, 35, 40, 41, 57], where the network’s parameters which are important to the past tasks are penalized when updating.”
Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, methodology): “by employing memory replay. Regularization approaches in order to alleviate catastrophic forgetting add an auxiliary term that penalizes changes in the weights when the model is trained on a new task [14,26,29,33,38,48,54,60]. Dynamic architectures would increase the number of neurons and network la”
A comprehensive, application-oriented study of catastrophic forgetting in DNNs (2019, methodology): “Principal recent approaches to avoid CF include ensemble methods (Ren et al., 2017; Fernando et al., 2017), dual-memory systems (Shin et al., 2017; Kemker & Kanan, 2017; Rebufﬁ et al., 2017; Gepperth & Karaoguz, 2016) and regularization approaches.”
RODEO: Replay for Online Object Detection (2020, neutral): “work sparsely updates weights to mitigate interference [8], 3) ensembling multiple classifiers together [9, 11, 43, 47, 53], and 4) rehearsal/replay models that store a subset of previous training inputs (or generate previous inputs) to mix with new examples when updating the network [4, 16, 17, 21,”
Continual Variational Autoencoder Learning via Online Cooperative Memorization (2022, methodology): “One of the most popular approaches is to use a regularization loss within the optimization procedure [14,21,23,26,36,41,47,52,56], where the network parameters which are important to the past learnt data are re-weighted when learning a new task, in order to attempt to preserve past knowledge.”
Lifelong Teacher-Student Network Learning (2021, methodology): “While deep learning approaches perform well in many specific data classification applications [4], they suffer from the catastrophic forgetting problem [5], [6], [7], [8] when”
Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process (2021, methodology): “Regularization methods introduce an auxiliary term in the loss function in order to penalize changes in the network’s weights when learning a new task [12, 15, 17, 23, 28, 38].”