Learning to Learn

By Jonathan Baxter. 1998

Preface. Part I: Overview Articles. 1. Learning to Learn: Introduction and Overview S. Thrun, L. Pratt. 2. A Survey of Connectionist Network Reuse Through Transfer L. Pratt, B. Jennings. 3. Transfer in Cognition A. Robins. Part II: Prediction. 4. Theoretical Models of Learning to Learn J. Baxter. 5. Multitask Learning R. Caruana. 6. Making a Low-Dimensional Representation Suitable for Diverse Tasks N. Intrator, S. Edelman. 7. The Canonical Distortion Measure for Vector Quantization and Function Approximation J. Baxter. 8. Lifelong Learning Algorithms S. Thrun. Part III: Relatedness. 9. The Parallel Transfer of Task Knowledge Using Dynamic Learning Rates Based on a Measure of Relatedness D.L. Silver, R.E. Mercer. 10. Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge S. Thrun, J. O'Sullivan. Part IV: Control. 11. CHILD: A First Step Towards Continual Learning M.B. Ring. 12. Reinforcement Learning with Self-Modifying Policies J. Schmidhuber, et al. 13. Creating Advice-Taking Reinforcement Learners R. Maclin, J.W. Shavlik. Contributing Authors. Index.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Federated Learning: Challenges, Methods, and Future Directions (2019, background): “There exists a large body of literature in machine learning that has modeled statistical heterogeneity via methods such as meta-learning [114] and multi-task learning [18, 37]; these ideas have been recently extended to the federated setting [24, 26, 35, 58, 106, 138].”
Set Transformer (2018, background): “Moreover, many meta-learning (Thrun & Pratt, 1998; Schmidhuber, 1987) problems which learn using different, but related tasks may also be treated as set-input tasks where an input set corresponds to the training dataset of a single task.”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Research in EPANNs will foster the adoption of such criteria as wider performance metrics for assessing lifelong learning capabilities (Thrun and Pratt, 2012; DARPA-L2M, 2017) of evolved plastic networks.”
Taskonomy: Disentangling Task Transfer Learning (2018, background): “compositional modeling [35, 10, 13, 23, 55, 92, 90], homomorphic cryptography [42], lifelong learning [93, 15, 85, 84], functional maps [71], certain aspects of Bayesian inference and Dirichlet processes [54, 91, 90, 89, 37, 39], few-shot learning [81, 25, 24, 70, 86], transfer learning [75, 84, 29,”
Efficient Representations for Lifelong Learning and Autoencoding (2014, neutral): “L G ] 4 D ec 2 01 4”
Open-world Learning and Application to Product Classification (2018, methodology): “e they do not do rejection of unseen classes as we do. They also require some training with each new class added. 5.2 Meta-learning Our work is clearly related to meta-learning (or learning to learn) [16], which has been successfully applied to many machine learning tasks lately. For example, it has”
Continuous Meta-Learning without Tasks (2019, background): “These fields all consider learning within a streaming series of tasks, wherein it is desirable to re-use information from previous tasks while avoiding negative transfer [12, 42].”
Adding New Tasks to a Single Network with Weight Trasformations using Binary Masks (2018, background): “This is crucial for increasing the knowledge of an intelligent system and developing eﬀective incremental [43,24], life-long learning [52,53,49] algorithms.”
Provable Guarantees for Gradient-Based Meta-Learning (2019, background): “Finally, we also provide an online-to-batch conversion result for task-averaged regret that implies good generalization guarantees when any of the variants of Ephemeral are run in the distributional LTL setting.”
Bounds for Linear Multi-Task Learning (2006, background): “Simultaneous learning of di¤erent tasks under some common constraint, often called multi-task learning, has been tested in practice with good results under a variety of di¤erent circumstances (see [ 5 ], [8], [17], [18]).”
Meta-Learning to Detect Rare Objects (2019, methodology): “Our approach falls under the umbrella of meta-learning [62, 63, 51, 52, 1, 50, 67, 70, 4, 42, 17, 58, 71, 22, 40, 34, 60, 28, 68, 53, 48, 8, 33], and is most related to the work on model parameter estimation [70].”
A Survey of Optimization Methods From a Machine Learning Perspective (2019, methodology): “In some meta learning methods, the optimization process itself can be regarded as a learning problem to learn the prediction gradient rather than a determined gradient descent algorithm [178].”
Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon (2018, methodology): “Meta-learning algorithms were ﬁrst introduced in the 1990s (Bengio et al., 1991; Schmidhuber, 1992; Thrun and Pratt, 1998) and have since then become particularly popular in ML, including, but not limited to, learning a gradient update rule (Hochreiter et al., 2001; Andrychowicz et al., 2016), few…”