Lifelong Learning Algorithms

By Sebastian Thrun. 1998

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “Meta-learning approaches include t as additional input [Schmidhuber, 1987, Thrun and Pratt, 1998, Thrun, 1998, Vilalta and Drissi, 2002].”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “In a broad sense, our approach is related to developmental learning [26, 16, 36] and lifelong learning [41, 25, 39, 29].”
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “This can be viewed as a form of transfer learning (Caruana, 1997; Thrun, 1998), where knowledge from pretraining tasks (e.”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Few-shot learning [25, 35, 46, 53] has recently been primarily addressed in the meta-learning framework [57, 71, 80], where a model learns an update rule for the parameters of a base-learner model [6, 7, 72] through a sequence of training episodes [79, 88].”
Meta-learning with differentiable closed-form solvers (2018, methodology): “Currently, most methods tackling few-shot learning operate within the general paradigm of meta-learning , which allows one to develop algorithms in which the process of learning can improve with the number of training episodes (Thrun, 1998; Vilalta & Drissi, 2002).”
Learning without Forgetting (2016, background): “Learning [24] and Never Ending Learning [25], are also”
Meta-Learning With Differentiable Convex Optimization (2019, methodology): “eNet [23], CIFAR-FS [3], and FC100 [20]. 2. Related Work Meta-learning studies what aspects of the learner (commonly referred to as bias or prior) effect generalization across a distribution of tasks [26,31,32]. Meta-learning approaches for few-shot learning can be broadly categorized these approach”
Knowledge Flow: Improve Upon Your Teachers (2019, background): “Other related work includes policy distillation (Rusu et al., 2016a), domain adaptation (Pan & Yang, 2010; Long et al., 2015; Tzeng et al., 2015) or lifelong learning (Chen & Liu, 2016; Thrun, 1998; Mitchell et al., 2015; Ruvolo & Eaton, 2013).”
Dynamic Few-Shot Visual Learning Without Forgetting (2018, background): “Meta-learning approaches typical involve a meta-learner model that given a few training examples of a new task it tries to quickly learn a learner model that “solves” this new task [20, 24, 1, 13, 19].”
Low-Shot Learning from Imaginary Data (2018, background): “More generally, a recent class of methods tries to frame low-shot learning itself as a “learning to learn” task, called meta-learning [33].”
TADAM: Task dependent adaptive metric for improved few-shot learning (2018, background): “learning a model that, given a task (set of labeled data), produces a classifier that generalizes across all tasks [31, 25].”
One-shot Learning with Memory-Augmented Neural Networks (2016, background): “However, previous work does suggest one potential strategy for attaining rapid learning from sparse data, and hinges on the notion of meta-learning (Thrun, 1998; Vi-lalta & Drissi, 2002).”
Gradient Episodic Memory for Continual Learning (2017, background): “Continual learning [Ring, 1994], also called lifelong learning [Thrun, 1994, Thrun and Pratt, 2012, Thrun, 1998, 1996], considers learning through a sequence of tasks, where the learner has to retain knowledge about past tasks and leverage that knowledge to quickly acquire new skills.”
Generative Negative Text Replay for Continual Vision-Language Pretraining (2022, background): “Continual Learning Continual learning [54,13,43,48] aims to integrate novel knowledge in a sequential fashion where old data are usually unavailable.”
Continual Sequence Generation with Adaptive Compositional Modules (2022, background): “Continual learning (CL) (Ring, 1998; Thrun, 1998) has been introduced to improve model’s ability to learn tasks in a stream by mitigating forgetting and facilitating knowledge transfer (Lopez-Paz and Ranzato, 2017), however, continual sequence generation is relatively under-investigated.”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, methodology): “However, these lifelong learning works mainly use the traditional learning methods such as regression (Ruvolo and Eaton, 2013), naive Bayes (Chen et al., 2015; Wang et al., 2019), and KNN (Thrun, 1998) to build a model for each task independently and hence there is no forgetting problem.”
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion (2021, background): “Continual learning models tackle the catastrophic forgetting of the old classes [67, 25].”