Learning Symbolic Models of Stochastic Domains
By Leslie Pack Kaelbling, Hanna M. Pasula, Luke S. Zettlemoyer. 2007
256 citations recorded by the source corpus.
In this article, we work towards the goal of developing agents that can learn to act in complex worlds. We develop a probabilistic, relational planning rule representation that compactly models noisy, nondeterministic action effects, and show how such rules can be effectively learned. Through experiments in simple planning domains and a 3D simulated blocks world with realistic physics, we demonstrate that this learning algorithm allows agents to effectively model world dynamics.
Subfields: Artificial Intelligence, Reinforcement Learning, Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Lihong Li
- Adam White
- Doina Precup
- Peter Stone
- Richard S. Sutton
- Hado van Hasselt
- Pieter Abbeel
In Reinforcement Learning
- Sergey Levine
- Adam White
- Lihong Li
- Doina Precup
- Shimon Whiteson
- Peter Stone
- Michael L. Littman
- Richard S. Sutton
In Probabilistic & Bayesian Methods
- Lihong Li
- Michael L. Littman
- Tomas Lozano-Perez
- Doina Precup
- Subramanian Ramamoorthy
- Kelsey R. Allen
- Carlos Diuk
- Richard S. Sutton
Citations in the corpus
- Neural Logic Machines (2019, methodology): “, 2017), symbolic models for learning in interactive domains (Pasula et al., 2007), structured task definition by object-oriented instructions (Denil et al.”
- Inventing Relational State and Action Abstractions for Effective and Efficient Bilevel Planning (2022, methodology): “In the course of our research, we considered many options inspired by prior work, including per-operator prediction error [12, 34], bisimulation [17, 29], and inverse planning-based objectives [35, 36], but found them all to be divergent from J real , leading to poor performance (see the baselines…”
- Exploring compact reinforcement-learning representations with linear regression (2009, methodology): “Specifically, we use KWIK linear regression (KWIK-LR) to learn the reward function in a factored MDP and the transition probabilities in domains encoded using Stochastic STRIPS [12] or Object Oriented MDPs (OOMDP) [5].”
- The Tools Challenge: Rapid Trial-and-Error Learning in Physical Problem Solving (2019, background): “However, work in robotics has developed approaches to planning in more abstract event spaces (Toussaint et al., 2018; Kaelbling, Pasula, & Zettlemoyer, 2007) that can be grounded to continuous physical states.”
- Learning spatial relationships between objects (2011, background): “…symbolic relationships between objects define the structure of an environment, and as a result, learning these relationships and being able to identify them in a static scene has important ramifications for planning and learning the effects of actions, as done in the work of Pasula et al. (2007).”
- Metric learning for generalizing spatial relations to new objects (2017, background): “Rather than relying on grounding existing symbols, other works have addressed learning symbols and effects of actions to abstract continuous states for the purpose of high-level planning [1, 2, 10, 14, 20].”
- Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction (2011, background): “Kaelbling et al. (2001) and Pasula et al. (2007) explored the learning of relational rule representations in stochastic domains.”
- A review of learning planning action models (2018, background): “In a series of works by (Pasula et al. (2004, 2007), Zettlemoyer et al. (2005)), the goal is to learn action models in noisy and stochastic domains.”