An object-oriented representation for efficient reinforcement learning

By Carlos Diuk, Andre Cohen, Michael L. Littman. 2008

355 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Investigating Contingency Awareness Using Atari 2600 Games (2012, methodology): “Diuk, Cohen, and Littman (2008) applied their DOORMAX algorithm to a restricted version of the game of Pitfall.”
The Arcade Learning Environment: An Evaluation Platform for General Agents (2012, methodology): “Diuk, Cohen, and Littman (2008) applied their DOORMAX algorithm to a restricted version of the game of Pitfall!.”
Neural Logic Machines (2019, methodology): “Examples include OO-MDP (Diuk et al., 2008; Kansky et al., 2017), symbolic models for learning in interactive domains (Pasula et al., 2007), structured task deﬁnition by object-oriented instructions (Denil et al., 2017), and structured policy learning (Garnelo et al., 2016).”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, methodology): “Echoing the pioneering work of Diuk et al. (2008) on the Atari 2600, we define the “salient event” to occur whenever the agent is located in the area rightwards from the starting screen.”
Toward Causal Representation Learning (2021, background): “Structured generative approaches further aim at decomposing an environment into multiple entities with causally correct relations among them, modulo the completeness of the variables, and confounding [15], [44], [59], [136], [264], [265].”
Towards Causal Representation Learning (2021, background): “among them, modulo the completeness of the variables, and confounding [58, 265, 43, 264, 14, 136].”
Causally Correct Partial Models for Reinforcement Learning (2020, methodology): “Within MBRL, commonly explored methods include actionconditional, next-step models (Oh et al., 2015; Ha & Schmidhuber, 2018; Chiappa et al., 2017; Schmidhuber, 2010; Xie et al., 2016; Deisenroth & Rasmussen, 2011; Lin & Mitchell, 1992; Li et al., 2015; Diuk et al., 2008; Igl et al., 2018; Ebert et a”
Learning First-Order Symbolic Representations for Planning from the Structure of the State Space (2019, background): “Object-oriented MDPs [13] and similar work in classical planning [37, 2, 1], build ﬁrst-order model representations but starting with a ﬁrst-order symbolic language, or with information about the actions and their arguments [11].”
Shaping Belief States with Generative Environment Models for RL (2019, background): “The idea of learning general world models to support decision-making is probably one of the most pervasive ideas in AI research, [30, 11, 41, 42, 14, 43, 44, 45, 46, 33, 30, 47, 48, 3, 49, 50, 51, 52].”
COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration (2019, methodology): “However, most existing approaches require hand-crafting object representations or their relations (Cobo et al., 2013; Diuk et al., 2008; Garnelo et al., 2016; Lázaro-Gredilla et al., 2019).”
Structured agents for physical construction (2019, background): “A main direction we explore is object-centric representations of the scene and agent’s actions (Diuk et al., 2008; Scholz et al., 2014), implemented with graph neural networks (Scarselli et al., 2009; Bronstein et al., 2017; Gilmer et al., 2017; Battaglia et al., 2018).”
Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks (2019, methodology): “…of objects have obtained objects either using ad hoc, task/environment speciﬁc object detection methods (Garnelo, Arulkumaran, and Shanahan 2016; Diuk, Cohen, and Littman 2008), or, in settings where it is possible, using object representations provided directly by the environment (Kansky et…”
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, methodology): “Inspired by developmental psychology literature [29] and object-oriented MDPs [7], we use entities or objects in the scene to parameterize goals in this environment.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “, 2001), object-oriented MDPs (Diuk et al., 2008), and MDPs described by linear temporal logic (Vaezipoor et al.”
Emergent Tool Use From Multi-Agent Autocurricula (2019, methodology): “[71] Carlos Diuk, Andre Cohen, and Michael L Littman.”
Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics (2017, neutral): “However, neither OO-MDPs nor R-MDPs deﬁne a transition function with an explicit OR of possible causes, and traditionally transition functions have not been learned in these models.”
Reasoning About Physical Interactions with Object-Oriented Prediction and Planning (2018, background): “Object-centric scene decompositions have also been considered as a potential state representation in reinforcement learning (Diuk et al., 2008; Scholz et al., 2014; Devin et al., 2017; Goel et al., 2018; Keramati et al., 2018).”
Unifying Task Specification in Reinforcement Learning (2016, background): “We explore different transition-based discounts in the taxi domain (Dietterich, 2000; Diuk et al., 2008).”