Lifelong Learning with Non-i.i.d. Tasks

By Anastasia Pentina, Christoph H. Lampert. 2015

106 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “When comparing with [33], our theoretical analysis does not rely on explicit task boundaries, which is a more realistic assumption for TFCL.”
Model Zoo: A Growing Brain That Learns Continually (2021, background): “…e.g., (Evgeniou et al., 2005; Cavallanti et al., 2010; Kumar and Daume III, 2012), and although such methods are popular in transfer learning (Pentina and Lampert, 2015; Jaakkola and Haussler, 1999), one cannot apply them in continual learning because we do not know the tasks beforehand.”
Task-Free Continual Learning (2018, background): “Inspired by biological systems, the ﬁeld of incremental learning, also referred to as continual learning or lifelong learning [24, 34, 37], aims at breaking this strong barrier between the training and testing phase.”
Memory Aware Synapses: Learning what (not) to forget (2017, background): “In this setting, lifelong learning (LLL) [24,35,37] comes as a natural solution.”
Encoder Based Lifelong Learning (2017, background): “The main challenge is to make the learned model adapt to new data from a similar or a different environment [18], without losing knowledge on the previously seen task(s).”
Continual Learning for Robotics (2019, background): “As in (Pentina and Lampert, 2015), we assume CL is tackling a PAC learnable problem in the approximation of a target hypothesis h∗ as well as learning from a sequence of non i.”
Explicit Inductive Bias for Transfer Learning with Convolutional Networks (2018, background): “In lifelong learning (Thrun & Mitchell 1995, Pentina & Lampert 2015) where a series of tasks is learned sequentially by a single model, the knowledge extracted from the previous tasks may be lost as new tasks are learned, resulting in what is known as catastrophic forgetting.”
Retrospective Adversarial Replay for Continual Learning (2022, background): “This has motivated research in Continual Learning (also referred to as Lifelong Learning and Incremental Learning), where a machine learning model learns from a stream of data coming from a succession of different tasks [29, 45, 66].”
Continual Learning in Neural Networks (2019, background): “The main challenge is to make the learned model adapt to new data from a similar or a different environment [113], without losing knowledge on the previously seen task(s).”
Local Rademacher Complexity-based Learning Guarantees for Multi-Task Learning (2016, background): “Generalization performance analysis in life-long learning has been investigated by [59, 13, 12, 55] and [54].”
A Continual Learning Survey: Defying Forgetting in Classification Tasks (2019, background): “The inﬂuence of dropout [43] and different activation functions on forgetting have been studied empirically for two sequential tasks [31] , and several works studied task incremental learning from a theoretical perspective [44], [45], [46], [47].”
Transfer Learning via Learning to Transfer (2018, background): “(Ruvolo & Eaton, 2013; Pentina & Lampert, 2015) consider lifelong learning as online meta-learning.”
On Localized Discrepancy for Domain Adaptation (2020, background): “Lifelong learning or learning to learn maximizes the overall performance on the tasks drawn from underlying task distribution Pentina and Lampert (2014, 2015); Khodak et al. (2019).”