Generalization to New Actions in Reinforcement Learning

By Ayush Jain, Andrew Szot, Joseph J. Lim. 2020

40 citations recorded by the source corpus.

A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances, such as making decisions from new action choices. However, standard reinforcement learning assumes a fixed set of actions and requires expensive retraining when given a new action set. To make learning agents more adaptable, we introduce the problem of zero-shot generalization to new actions. We propose a two-stage framework where the agent first infers action representations from action information acquired separately from the task. A policy flexible to varying action sets is then trained with generalization objectives. We benchmark generalization on sequential tasks, such as selecting from an unseen tool-set to solve physical reasoning puzzles and stacking towers with novel 3D shapes. Videos and code are available at this https URL

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “, 2016), enable generalization to unseen (Ayush et al., 2020) or under-explored (Chandak et al.”
Learning to Synthesize Programs as Interpretable and Generalizable Policies (2021, background): “[86] Ayush Jain, Andrew Szot, and Joseph J Lim.”
Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, methodology): “The CREATE environment (Jain et al., 2020) is a challenging physical reasoning benchmark with a large variety of tools as actions and also supports evaluation with unseen actions.”