Object focused q-learning for autonomous agents

By Luis C. Cobo, Charles Lee Isbell Jr., Andrea Lockerd Thomaz. 2013

29 citations recorded by the source corpus.

We present Object Focused Q-learning (OF-Q), a novel reinforcement learning algorithm that can offer exponential speed-ups over classic Q-learning on domains composed of independent objects. An OF-Q agent treats the state space as a collection of objects organized into different object classes. Our key contribution is a control policy that uses non-optimal Q-functions to estimate the risk of ignoring parts of the state space. We compare our algorithm to traditional Q-learning and previous arbitration algorithms in two domains, including a version of Space Invaders.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. J. Urgen Schmidhuber
3. Mark O. Riedl
4. Alexander Lerchner
5. Loic Matthey
6. Josh Tenenbaum
7. Karthik Narasimhan
8. Christopher P. Burgess

In Computational Theory and Mathematics

1. Karthik Narasimhan
2. Christopher P. Burgess
3. Andrea Lockerd Thomaz
4. Josh Tenenbaum
5. Charles Lee Isbell Jr.
6. Tejas D. Kulkarni
7. Alexander Lerchner
8. Loic Matthey

In Information Systems

1. Karthik Narasimhan
2. Andrea Lockerd Thomaz
3. Josh Tenenbaum
4. Charles Lee Isbell Jr.
5. Tejas D. Kulkarni
6. Emma Brunskill
7. Mark O. Riedl

Citations in the corpus

• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration (2019, methodology): “However, most existing approaches require hand-crafting object representations or their relations (Cobo et al., 2013; Diuk et al., 2008; Garnelo et al., 2016; Lázaro-Gredilla et al., 2019).”
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, background): “Object-based Reinforcement Learning Object-based representations [7, 4] that can exploit the underlying structure of a problem have been proposed to alleviate the curse of dimensionality in RL.”
• Learning From Explanations Using Sentiment and Advice in RL (2017, methodology): “While natural language explanations can be incorporated into many machine learning algorithms, we used OF-Q to represent tasks for two main reasons [5].”
• Strategic Object Oriented Reinforcement Learning (2018, methodology): “Model free methods to use object representation [9, 23, 5] fail to scale to large MDPs and do not leverage object”