Continual learning in reinforcement environments

By Mark B. Ring. 1995

Subfields: Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “5 Deep Hierarchical RL (HRL) and Subgoal Learning with FNNs and RNNs Multiple learnable levels of abstraction (Fu, 1977; Lenat and Brown, 1984; Ring, 1994; Bengio et al., 2013; Deng and Yu, 2014) seem as important for RL as for SL.”
Gradient Episodic Memory for Continual Learning (2017, background): “Continual learning [Ring, 1994], also called lifelong learning [Thrun, 1994, Thrun and Pratt, 2012, Thrun, 1998, 1996], considers learning through a sequence of tasks, where the learner has to retain knowledge about past tasks and leverage that knowledge to quickly acquire new skills.”
PathNet: Evolution Channels Gradient Descent in Super Neural Networks (2017, methodology): “The framework for AGI described above includes aspects of transfer learning [22], continual learning [16] and multitask learning [2].”
Progress & Compress: A scalable framework for continual learning (2018, background): “Copyright 2018 by the author(s). previously trained problems is known as continual learning (e.g. Ring, 1995).”
Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer (2022, background): “Such a learning paradigm is known as continual learning (CL) [5, 23], which has recently attracted much attention.”
Architecture Matters in Continual Learning (2022, background): “Continual learning (CL) (Ring, 1995; Thrun, 1995) is a branch of machine learning where the model is exposed to a sequence of tasks with the hope of exploiting existing knowledge to adapt quickly to new tasks.”
Wide Neural Networks Forget Less Catastrophically (2021, background): “To overcome these inefﬁciencies, ﬁelds, such as continual learning (CL) (Ring, 1995) or lifelong learning (Thrun, 1995) are gaining a lot of attention recently.”
Lifelong Intent Detection via Multi-Strategy Rebalancing (2021, background): “Recently, lifelong learning has received increasing attention and is considered to be the most promising approach to address this problem [19, 22].”
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution (2021, background): “This setting, dubbed in the literature as continual learning (Ring, 1994; Rebuffi et al., 2017; Kirkpatrick et al., 2017; Lopez-Paz and Ranzato, 2017), introduces new challenges not found in the traditional multi-task setup, most famously catastrophic forgetting (McCloskey and Cohen, 1989), in which”
Efficient Continual Learning with Modular Networks and Task-Driven Priors (2020, background): “Continual Learning (CL) is a learning framework whereby an agent learns through a sequence of tasks (Ring, 1994; Thrun, 1994; 1998), observing each task once and only once.”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, background): “Continual learning1 (Ring, 1994) is a machine learning paradigm, whose objective is to adaptively learn across time by leveraging previously learned tasks to improve generalization for future tasks.”
Variational Auto-Regressive Gaussian Processes for Continual Learning (2020, background): “Continual Learning (CL) is the constant development of complex behaviors by building upon previously acquired skills (Ring, 1994; Thrun, 1998); humans and other animals exhibit knowledge acquisition for continual skill development (Hoppitt & Laland, 2013).”
Continual Reinforcement Learning with Multi-Timescale Replay (2020, background): “…agents that are deployed in the real world have to be able to learn from data streaming in from a nonstationary distribution and incrementally build on their knowledge over time, while operating with limited computational resources; this is the challenge of continual learning [Ring, 1994].”
Continual Learning with Adaptive Weights (CLAW) (2019, background): “A continual learner must be able to learn a new task, crucially, without forgetting previous tasks (Ring, 1995; Srivastava et al., 2013; Schwarz et al., 2018; Serra et al., 2018; Hu et al., 2019).”
Continual Reinforcement Learning in 3D Non-stationary Environments (2019, background): “Despite the the obvious interest in applying CL to less supervised settings and the early, promising results in this context [40, 48], reinforcement learning tasks constitute a much more complex challenge where it is generally more difficult to disentangle the complexity introduced by distributional”
Functional Regularisation for Continual Learning using Gaussian Processes (2019, background): “…learning problem has recently received increased attention, following the influential work on Elastic Weight consolidation (EWC) (Kirkpatrick et al., 2017), although work on the problem dates back far longer than this (e.g. Ring, 1994; Robins, 1995; Schmidhuber, 2013; Goodfellow et al., 2013).”
Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference (2018, background): “In this paper we explore a variant called continual learning (Ring, 1994).”
Scalable Recollections for Continual Lifelong Learning (2017, background): “Continual learning (Thrun, 1994; Ring, 1994; Thrun, 1996; Thrun & Pratt, 1998; 2012) has three main requirements: (1) continually learn in a non-stationary environment, (2) retain memories which are useful, (3) manage time and memory resources over a long period of time.”
Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation (2022, background): “We also report metrics for two settings in LLL following (Mi et al., 2020): where T is the total number of LLL tasks; Ω all,i is the average test performance on all the previous tasks after the i th task has been learned; Ω first,i is performance on the ﬁrst task after the i th task has been learned”