Learning to Learn: Introduction and Overview

By Sebastian Thrun, L. Pratt. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Few-shot Learning with Retrieval Augmented Language Models (2022, background): “Few-shot learning, the task of learning from very few examples, has been studied for decades (Thrun & Pratt, 1998; Fink, 2005; Vinyals et al., 2016), but has recently seen an explosion of interest in NLP with the arrival of large pre-trained models, which exhibit emergent few-shot learning…”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “Meta-learning approaches include t as additional input [Schmidhuber, 1987, Thrun and Pratt, 1998, Thrun, 1998, Vilalta and Drissi, 2002].”
How to train your MAML (2018, background): “Meta-learning (Schmidhuber, 1987; Vilalta & Drissi, 2002), or learning to learn (Thrun & Pratt, 1998), can instead be used to automatically learn across-task knowledge usually referred to as across-task (or sometimes slow) knowledge such that our model can, at inference time, quickly acquire…”
Meta-Learning with Latent Embedding Optimization (2018, methodology): “The problem of few-shot adaptation has been approached in the context of fast weights (Hinton & Plaut, 1987; Ba et al., 2016), learning-to-learn (Schmidhuber, 1987; Thrun & Pratt, 1998; Hochre-iter et al., 2001; Andrychowicz et al., 2016), and through meta-learning .”
Learning to reinforcement learn (2016, methodology): “The second case has been explored in the machine learning literature under the rubric of meta-learning (Schmidhuber et al., 1996; Thrun and Pratt, 1998).”
Meta-Learning with Temporal Convolutions (2017, background): “Pioneered by Thrun and Pratt [26], Schmidhuber [22], Naik and Mammone [19], meta-learning is not a new idea.”
Pre-Trained Models: Past, Present and Future (2021, background): “One milestone for this issue is the introduction of transfer learning (Thrun and Pratt, 1998; Pan and Yang, 2009).”
Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, background): “Metalearning [42] is the idea of learning to learn.”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, background): “+ forward transfer – no backward transfer – no knowledge retention – off-line learning (Thrun and Pratt, 1998) (Finn et al.”
Scalable Recollections for Continual Lifelong Learning (2017, background): “Continual learning (Thrun, 1994; Ring, 1994; Thrun, 1996; Thrun & Pratt, 1998; 2012) has three main requirements: (1) continually learn in a non-stationary environment, (2) retain memories which are useful, (3) manage time and memory resources over a long period of time.”
Machine Theory of Mind (2018, methodology): “We consider the challenge of building a Theory of Mind as essentially a meta-learning problem (Schmidhuber et al., 1996; Thrun & Pratt, 1998; Hochreiter et al., 2001; Vilalta & Drissi, 2002).”
Meta-Transfer Learning for Few-Shot Learning (2018, methodology): “In this section, we focus on methods using the supervised meta-learning paradigm [12, 51, 9] most relevant to ours and compared to in the experiments.”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “ogically plausible learning rules. Schmidhuber et al.continued to explore self-referential systems and meta-learning in subsequent work [49], [50]. S. Thrun et al.coined the term learning to learn in [8] as an alternative to meta-learning and proceeded to explore and dissect available literature in ”
Few-Shot Learning via Learning the Representation, Provably (2020, background): “The idea of multitask representation learning at least dates back to Caruana (1997), Thrun and Pratt (1998), Baxter (2000).”
Meta-Learning Improves Lifelong Relation Extraction (2019, background): “Meta-Learning Meta-learning, or learning to learn (Schmidhuber, 1987; Naik and Mam-mone, 1992; Thrun and Pratt, 1998), aims to develop algorithms that learn a generic knowledge of how to solve tasks from a given distribution of tasks, by generalizing from solving related tasks from that…”
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning (2018, background): “Our problem setting relates to meta-learning, a long-standing problem of interest in machine learning that is concerned with enabling artificial agents to efficiently learn new tasks by learning to learn (Thrun and Pratt, 1998; Schmidhuber and Huber, 1991; Naik and Mammone, 1992; Lake et al., 2015).”
Meta-Transfer Learning Through Hard Tasks (2019, background): “Research literature on few-shot learning exhibits great diversity, spanning from data augmentation [11]–[14] to supervised metalearning [18], [37].”
An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “ta-learning. Research literature on few-shot learning paradigms exhibits a high diversity from using data augmentation techniques [60, 62] over sharing feature representation [3, 61] to meta-learning [16, 58]. In this paper, we focus on the meta-learning paradigm that leverages few-shot learning exp”
Meta-Learning with Warped Gradient Descent (2019, background): “Early work focused on evolutionary approaches (Schmidhuber, 1987; Bengio et al., 1991; Thrun & Pratt, 1998).”