Using Imagery to Simplify Perceptual Abstraction in Reinforcement Learning Agents
By Samuel Wintermute. 2010
20 citations recorded by the source corpus.
In this paper, we consider the problem of reinforcement learning in spatial tasks. These tasks have many states that can be aggregated together to improve learning efficiency. In an agent, this aggregation can take the form of selecting appropriate perceptual processes to arrive at a qualitative abstraction of the underlying continuous state. However, for arbitrary problems, an agent is unlikely to have the perceptual processes necessary to discriminate all relevant states in terms of such an abstraction.To help compensate for this, reinforcement learning can be integrated with an imagery system, where simple models of physical processes are applied within a low-level perceptual representation to predict the state resulting from an action. Rather than abstracting the current state, abstraction can be applied to the predicted next state. Formally, it is shown that this integration broadens the class of perceptual abstraction methods that can be used while preserving the underlying problem. Empirically, it is shown that this approach can be used in complex domains, and can be beneficial even when formal requirements are not met.
Subfields: Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Reinforcement Learning
- Marc G. Bellemare
- Matthew J. Hausknecht
- Michael H. Bowling
- Sergey Levine
- Peter Stone
- Pieter Abbeel
- Joel Veness
- Joel Lehman
Citations in the corpus
- Investigating Contingency Awareness Using Atari 2600 Games (2012, methodology): “Wintermute (2010) proposed a method that also extracts objects from the displayed image and embeds them into a logic-based architecture , SOAR.”
- The Arcade Learning Environment: An Evaluation Platform for General Agents (2012, methodology): “Wintermute (2010) proposed a method that also extracts objects from the displayed image and embeds them into a logicbased architecture, SOAR.”
- Sketch-Based Linear Value Function Approximation (2012, methodology): “Different learning methods have been evaluated on the Atari 2600 platform [9, 26, 12].”
- Bridging the Gap: Converting Human Advice into Imagined Examples (2019, background): “Perhaps the work most related to our approach are imagery (Wintermute, 2010) and imagined trajectories (Weber et al.”
- A Neuroevolution Approach to General Atari Game Playing (2014, background): “Research in this area has focused on learning to play single games using perceptual imagery [52], human demonstration [9], discovery of objective functions [48], and use of raw visual inputs [30], [34], [35], [49].”