Composable Planning with Attributes

By Amy Zhang, Adam Lerer, Sainbayar Sukhbaatar, Rob Fergus, Arthur Szlam. 2018

68 citations recorded by the source corpus.

The tasks that an agent will need to solve often are not known during training. However, if the agent knows which properties of the environment are important then, after learning how its actions affect those properties, it may be able to use this knowledge to solve complex tasks without training specifically for them. Towards this end, we consider a setup in which an environment is augmented with a set of user defined attributes that parameterize the features of interest. We propose a method that learns a policy for transitioning between "nearby" sets of attributes, and maintains a graph of possible transitions. Given a task at test time that can be expressed in terms of a target set of attributes, and a current state, our model infers the attributes of the current state and searches over paths through attribute space to get a high level plan, and then uses its low level policy to execute the plan. We show in 3D block stacking, grid-world games, and StarCraft that our model is able to generalize to longer, more complex tasks at test time by composing simpler learned policies.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Structured agents for physical construction (2019, background): “…an unstable stack (Hamrick et al., 2018), learning basic block-stacking by predicting shortest paths between current and goal states via a transition model (Zhang et al., 2018), and learning object representations and coarse-grained physics models for stacking blocks (Janner et al., 2019).”
Relational inductive biases, deep learning, and graph networks (2018, background): “, 2018), developing model-based approaches with an emphasis on abstraction (Kansky et al., 2017; Konidaris et al., 2018; Zhang et al., 2018; Hay et al., 2018), investing more heavily in meta-learning (Wang et al.”
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning (2019, methodology): “Similar algorithms have been proposed for visual navigation (Anderson et al., 2018; Gupta et al., 2017; Mirowski et al., 2016; Zhang et al., 2018; Zhu et al., 2017).”
Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, methodology): “In contrast to these parametric methods, several non-parametric approaches can be adopted, such as episodic memory [69], transition graphs [70], and Gaussian Processes [71].”
Learning Features and Abstract Actions for Computing Generalized Plans (2018, methodology): “In this sense, the work is different than action model learning (Yang, Wu, and Jiang 2007) and model-based reinforcement learning and closer in aims to work on learning general policies from examples or experience (Martı́n and Geffner 2004; Fern, Yoon, and Givan 2006; Sukhbaatar et al. 2015; Zhang e”
Neural Topological SLAM for Visual Navigation (2020, background): “Others build a topological representation with explicit semantics [42, 46], which limits tasks and environments that can be tackled.”
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation (2022, background): “…language generation applications such as story generation (Fan et al., 2018), contextual text completion (Radford et al., 2019), dialog modeling (Zhang et al., 2018), etc. Text completion models take a text passage or prefix wj0 ∼ o as an input and generate a coherent continuation of the…”
Modeling Long-horizon Tasks as Sequential Interaction Landscapes (2020, methodology): “As an example, Zhang et al. [60] propose to use user-deﬁned attributes in environments to then learn policies that enable transitioning between these features of interest.”
Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planning (2020, background): “This also allows the embedding of policies in classical motion planning structures [23, 24, 25, 26, 27, 28, 29, 30].”
Bayesian Relational Memory for Semantic Visual Navigation (2019, methodology): “From a reinforcement learning perspective, our work is related to the model-based approaches [14, 45, 61, 44, 32], in the sense that we model the environment layout via a re-”
Learning to Visually Navigate in Photorealistic Environments Without any Supervision (2020, unknown): “Finally, our approach is most related to a recent line of research that uses multiple stages of learning to build a set or graph of scene observations [10, 26, 27, 35].”