Is Learning The n-th Thing Any Easier Than Learning The First?

By Sebastian Thrun. 1995

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Multi-task Sequence to Sequence Learning (2015, background): “Such framework has been widely studied by Thrun (1996); Caruana (1997); Evgeniou & Pontil (2004); Ando & Zhang (2005); Argyriou et al. (2007); Kumar & III (2012), among many others.”
Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, background): “, nearest neighbors) that increase capacity when memorizing new data [40, 42], our developmental network cumulatively grows its capacity from novel tasks.”
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition (2013, background): “Learning from related tasks also has a long history in machine learning beginning with Caruana (1997) and Thrun (1996).”
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations (2021, background): “The usual setup for few-shot out-of-domain transfer learning consists of transfer learning from a “parent” task, with abundant training examples, to a “child” task that only has a few training examples (Thrun, 1996; Ravi and Larochelle, 2017).”
Segment Anything (2023, methodology): “Pre-trained models have been adapted to downstream tasks since the early days of machine learning [99].”
Meta-learning with differentiable closed-form solvers (2018, background): “Another seminal work is the one of Thrun (1996), which presents the so-called lifelong learning scenario, where a learning algorithm gradually encounters an ordered sequence of learning problems.”
Kernels for Vector-Valued Functions: a Review (2011, background): “These ideas are related to transfer learning [97, 20, 12, 74], a term which refers to systems that learn by transferring knowledge between different domains, for example: “what can we learn about running through seeing walking?” More formally, the classical supervised learning problem requires estim”
Low-Shot Learning from Imaginary Data (2018, neutral): “Low-shot learning is a classic problem [32].”
Gradient Episodic Memory for Continual Learning (2017, background): “Continual learning [Ring, 1994], also called lifelong learning [Thrun, 1994, Thrun and Pratt, 2012, Thrun, 1998, 1996], considers learning through a sequence of tasks, where the learner has to retain knowledge about past tasks and leverage that knowledge to quickly acquire new skills.”
Selective Experience Replay for Lifelong Learning (2018, background): “Lifelong machine learning (Thrun 1996) examines learning multiple tasks in sequence, with an emphasis on how previous knowledge of different tasks can be used to improve the training time and learning of current tasks.”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, background): “In contrast, the lifelong learning (LL) paradigm (Thrun, 1996; Chen & Liu, 2018; Parisi et al., 2019) deﬁnes a biologically inspired learning approach where models learn tasks in sequence, ideally preserving past knowledge and leveraging it to efﬁ-ciently learn new tasks.”
Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, background): “The empirical results in Figure 1 show that when trained with noisy labels, the model becomes much more prone to catastrophic forgetting [20, 57, 78, 67].”
Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data (2021, background): “Learning efficacy entails the simple question, “is learning the n-th thing easier than learning the first thing?” [31].”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, background): “…1989; Chaudhry et al., 2018), sequential learning (McCloskey and Cohen, 1989; Shin et al., 2017; Aljundi et al., 2019), explanationbased learning (Thrun, 1996), and never-ending learning (Carlson et al., 2010). backward transfer deteriorates the performance on previous tasks (if high, it enables…”
Knowledge Distillation for Incremental Learning in Semantic Segmentation (2019, background): “Catastrophic forgetting has been faced even before the rise of neural networks popularity [27, 28, 29] and more recently has been rediscovered and tackled in different ways.”
Incremental Learning Techniques for Semantic Segmentation (2019, background): “It has been addressed even before the rise of neural networks popularity [6, 25, 33], but more recently it has been rediscovered and tackled in different ways.”
Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference (2018, background): “Such agents must incrementally learn and adapt to a changing environment while maintaining memories of what they have learned before, a setting known as lifelong learning (Thrun, 1994; 1996).”
Scalable Recollections for Continual Lifelong Learning (2017, background): “Continual learning (Thrun, 1994; Ring, 1994; Thrun, 1996; Thrun & Pratt, 1998; 2012) has three main requirements: (1) continually learn in a non-stationary environment, (2) retain memories which are useful, (3) manage time and memory resources over a long period of time.”