Learning Curriculum Policies for Reinforcement Learning

By Sanmit Narvekar, Peter Stone. 2018

100 citations recorded by the source corpus.

Curriculum learning in reinforcement learning is a training methodology that seeks to speed up learning of a difficult target task, by first training on a series of simpler tasks and transferring the knowledge acquired to the target task. Automatically choosing a sequence of such tasks (i.e., a curriculum) is an open problem that has been the subject of much recent work in this area. In this paper, we build upon a recent method for curriculum design, which formulates the curriculum sequencing problem as a Markov Decision Process. We extend this model to handle multiple transfer learning algorithms, and show for the first time that a curriculum policy over this MDP can be learned from experience. We explore various representations that make this possible, and evaluate our approach by learning curriculum policies for multiple agents in two different domains. The results show that our method produces curricula that can train agents to perform on a target task as fast or faster than existing methods.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

From Few to More: Large-scale Dynamic Multiagent Curriculum Learning (2019, methodology): “Narvekar et al. (2016) ﬁrstly applied CL to RL and proposed a new CL framework.”
Automated curricula through setter-solver interactions (2019, unknown): “Some approaches work for a set of pre-speciﬁed tasks (Narvekar & Stone, 2019), but they require human effort to hand-select tasks from this set.”
Autonomous Reinforcement Learning via Subgoal Curricula (2021, background): “Prior works have shown that using a curriculum can enable faster learning and improve performance [9, 10, 40, 30, 33, 27].”
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020, background): “perts sequence task model automatic both asymptotic conference No Restrictions (Section 4.2.4) Narvekar et al. (2017) automatic sequence value function automatic adaptive time to threshold conference Narvekar and Stone (2019) domain experts sequence value function, shaping reward automatic adaptive ”