Rapid Task-Solving in Novel Environments

By Samuel Ritter, Laurent Sartran, Adam Santoro, Matthew Botvinick, David Raposo. 2020

31 citations recorded by the source corpus.

When thrust into an unfamiliar environment and charged with solving a series of tasks, an effective agent should (1) leverage prior knowledge to solve its current task while (2) efficiently exploring to gather knowledge for use in future tasks, and then (3) plan using that knowledge when faced with new tasks in that same environment. We introduce two domains for conducting research on this challenge, and find that state-of-the-art deep reinforcement learning (RL) agents fail to plan in novel environments. We develop a recursive implicit planning module that operates over episodic memories, and show that the resulting deep-RL agent is able to explore and plan in novel environments, outperforming the nearest baseline by factors of 2-3 across the two domains. We find evidence that our module (1) learned to execute a sensible information-propagating algorithm and (2) generalizes to situations beyond its training experience.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

Towards mental time travel: a hierarchical memory for reinforcement learning agents (2021, methodology): “Figure 2: We evaluated our model across a wide variety of task domains, some of which are modified or used from other recent memory papers [19, 21, 2, 48].”
Transformers are Meta-Reinforcement Learners (2022, background): “Finally, Ritter et al. (2021) also applied multi-head self-attention for rapid task solving for RL environments.”