Learning Generalized Reactive Policies using Deep Neural Networks

By Edward Groshev, Aviv Tamar, Siddharth Srivastava, Pieter Abbeel. 2017

133 citations recorded by the source corpus.

We present a new approach to learning for planning, where knowledge acquired while solving a given set of planning problems is used to plan faster in related, but new problem instances. We show that a deep neural network can be used to learn and represent a generalized reactive policy (GRP) that maps a problem instance and a state to an action, and that the learned GRPs efficiently solve large classes of challenging problem instances. In contrast to prior efforts in this direction, our approach significantly reduces the dependence of learning on handcrafted domain knowledge or feature selection. Instead, the GRP is trained from scratch using a set of successful execution traces. We show that our approach can also be used to automatically learn a heuristic function that can be used in directed search algorithms. We evaluate our approach using an extensive suite of experiments on two challenging planning problem domains and show that our approach facilitates learning complex decision making policies and powerful heuristic functions with minimal human input.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Learning First-Order Symbolic Representations for Planning from the Structure of the State Space (2019, background): “Deep reinforcement learning (DRL) methods [30], on the other hand, generate policies over high-dimensional perceptual spaces like images, without using any prior symbolic knowledge [19, 10, 15].”
Deep Reactive Policies for Planning in Stochastic Nonlinear Domains (2019, background): “Currently, most works on DRPs in model-known planning have focused on discrete stochastic policies (Issakkimuthu, Fern, and Tadepalli 2018; Toyer et al. 2018; Groshev et al. 2018).”
Value Propagation Networks (2018, methodology): “Model-based planning with end-to-end architectures has recently shown promising results on a variety of tasks and environments, often using Deep Reinforcement Learning as the algorithmic framework (Silver et al., 2016; Oh et al., 2017; Weber et al., 2017; Groshev et al., 2017; Farquhar et al., 2017;”
Transfer of Deep Reactive Policies for MDP Planning (2018, background): “There also has been an early attempt on transfer in planning problems, for two specific classical (deterministic) domains of TSP and Sokoban [Groshev et al., 2018].”
Model-free, Model-based, and General Intelligence (2018, background): “The key difference between DL and DRL is in the way in which the functions fθ are learned during training.”