LTL2Action: Generalizing LTL Instructions for Multi-Task RL

By Pashootan Vaezipoor, Andrew C. Li, Rodrigo Toro Icarte, Sheila A. McIlraith. 2021

113 citations recorded by the source corpus.

We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. Instructions are expressed in a well-known formal language -- linear temporal logic (LTL) -- and can specify a diversity of complex, temporally extended behaviours, including conditionals and alternative realizations. Our proposed learning approach exploits the compositional syntax and the semantics of LTL, enabling our RL agent to learn task-conditioned policies that generalize to new instructions, not observed during training. To reduce the overhead of learning LTL semantics, we introduce an environment-agnostic LTL pretraining scheme which improves sample-efficiency in downstream environments. Experiments on discrete and continuous domains target combinatorial task sets of up to $\sim10^{39}$ unique tasks and demonstrate the strength of our approach in learning to solve (unseen) tasks, given LTL instructions.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “, 2008), and MDPs described by linear temporal logic (Vaezipoor et al., 2021; Jiang et al., 2021a).”
Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning (2020, methodology): “A popular approach was to translate the LTL specification into a finite state machine, reward the agent when the machine hits an accepting state, and learn policies using the cross-product baseline.”