Active Task Selection for Lifelong Machine Learning

By Paul Ruvolo, Eric Eaton. 2013

77 citations recorded by the source corpus.

In a lifelong learning framework, an agent acquires knowledge incrementally over consecutive learning tasks, continually building upon its experience. Recent lifelong learning algorithms have achieved nearly identical performance to batch multi-task learning methods while reducing learning time by three orders of magnitude. In this paper, we further improve the scalability of lifelong learning by developing curriculum selection methods that enable an agent to actively select the next task to learn in order to maximize performance on future learning tasks. We demonstrate that active task selection is highly reliable and effective, allowing an agent to learn high performance models using up to 50% fewer tasks than when the agent has no control over the task order. We also explore a variant of transfer learning in the lifelong learning setting in which the agent can focus knowledge acquisition toward a particular target task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Selective Experience Replay for Lifelong Learning (2018, background): “The order in which tasks are encountered does impact learning, and several groups have investigated the effects of ordering (Bengio et al. 2009; Ruvolo and Eaton 2013a; Narvekar et al. 2016).”
Learning Cumulatively to Become More Knowledgeable (2016, background): “[29] further enhanced ELLA through actively selection of the next task to learn.”
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, background): “Lifelong and continual learning have been examined in both the supervised setting (Ruvolo and Eaton, 2013a) and the reinforcement learning setting (Ring, 1997; Ammar et al., 2014).”
Lifelong-RL: Lifelong Relaxation Labeling for Separating Entities and Aspects in Opinion Targets (2016, background): “Traditional LML focuses on supervised learning (Thrun, 1998; Ruvolo and Eaton, 2013; Chen et al., 2015).”
Asymmetric multi-task learning based on task relatedness and loss (2016, background): “Ruvolo & Eaton (2013) propose a curriculum learning method based on an active task selection, which aims to maximize the task diversity to learn better approximating latent parameter bases shared across all tasks.”
Source Task Creation for Curriculum Learning (2016, background): “In particular, Ruvolo and Eaton [16] show how a learner can actively select tasks to improve learning speed for all tasks, or for a specific target task.”
Tensor Based Knowledge Transfer Across Skill Categories for Robot Control (2017, background): “The choice of task sequence is important to the outcome in lifelong learning [ Bengio et al. , 2009 ] , but despite some work [ Ruvolo and Eaton, 2013a ] , how to predict a good curriculum in advance is still an open question.”
Deep Bayesian Unsupervised Lifelong Learning (2021, background): “For example, different tasks can represent different subsets of classes or labels in the same supervised learning problem [6] and can also represent supervised learning problems in different fields, where researchers target to perform continual learning across different domains [7] or lifelong trans”
Active Lifelong Learning With "Watchdog" (2018, methodology): “(Ruvolo and Eaton 2013a) concludes: Random (ELLARand): the next task arrives randomly; Information Maximization (ELLA-Info): the next task should be maximize the expected information gain over the knowledge library L; Diversity (ELLA-Diver): the next task is chosen as the one that current library L…”
Multi-task Learning with Labeled and Unlabeled Tasks (2016, methodology): “The idea of choosing tasks was used in active curriculum selection [38, 36], where the learner can influence the order in which tasks are processed.”