Learning to Compose Skills

By Himanshu Sahni, Saurabh Kumar, Farhan Tejani, Charles Lee Isbell Jr.. 2017

44 citations recorded by the source corpus.

We present a differentiable framework capable of learning a wide variety of compositions of simple policies that we call skills. By recursively composing skills with themselves, we can create hierarchies that display complex behavior. Skill networks are trained to generate skill-state embeddings that are provided as inputs to a trainable composition function, which in turn outputs a policy for the overall task. Our experiments on an environment consisting of multiple collect and evade tasks show that this architecture is able to quickly build complex skills from simpler ones. Furthermore, the learned composition function displays some transfer to unseen combinations of skills, allowing for zero-shot generalizations.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “(Sahni et al., 2017; Barreto et al., 2019) have focused on the natural combination of both skill and composition to allow for explicit reuse of previously acquired knowledge in the form of skills.”
Modular Networks: Learning to Decompose Neural Computation (2018, background): “There is also a wide range of literature in robotics that uses modularity to learn robot skills more efficiently by reusing functionality shared between tasks [19, 6].”
Multi-Task Reinforcement Learning with Soft Modularization (2020, methodology): “Besides focusing on the gradients, another direction is to design compositional model with multiple modules (Singh, 1992; Devin et al., 2017; Andreas et al., 2017; Rusu et al., 2016; Qureshi et al., 2020; Peng et al., 2019; Haarnoja et al., 2018a; Sahni et al., 2017).”
Learning Abstract Options (2018, background): “[23] leverages a logic for combining pre-learned skills by learning an embedding to represent the combination of a skill and state.”
Fast Adaptation to New Environments via Policy-Dynamics Value Functions (2020, background): “A large body of work proposes the use of learned skill and task embeddings for transfer in RL (Da Silva et al., 2012; Sahni et al., 2017; Oh et al., 2017; Gupta et al., 2017; Hausman et al., 2018; He et al., 2018).”
Memorize or generalize? Searching for a compositional RNN in a haystack (2018, background): “A very promising recent work (Sahni et al., 2017) focuses on skill composition.”