Reinforcement learning with selective perception and hidden state

By Andrew McCallum, Dana H. Ballard. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policy (2019, methodology): “The UTree algorithm [McCallum and Ballard, 1996; Uther and Veloso, 1998] learns a tree, starting with a single abstract state and then continually splitting as appropriate (based on a splitting criterion), ending when a stopping criterion is reached.”
A Distributional Perspective on Reinforcement Learning (2017, background): “McCallum (1995), for example, showed the importance of coupling representation learning with policy learning in partially observable domains.”
Reinforcement Learning: A Survey (1996, positive): “It outperformed Q-learning with backpropagation in a simple video-game environment and was used by McCallum (1995) (in conjunction with other techniques for dealing with partial observability) to learn behaviors in a complex driving-simulator.”
An MDP-Based Recommender System (2002, positive): “And, in fact, there are methods for learning the state representation (e.g., McCallum, 1996 ).”
Closing the learning-planning loop with predictive state representations (2009, methodology): “In one instance [18], researchers obtained a POMDP heuristically from the output of a model-free algorithm [13] and demonstrated planning on a small toy maze.”
Deep Variational Reinforcement Learning for POMDPs (2018, background): “The exponential growth in t of H t can be addressed, e.g., with sufﬁx trees (McCallum & Ballard, 1996; Shani et al., 2005; Bellemare et al., 2014; Bellemare, 2015; Messias & Whiteson, 2017).”
Tree based hierarchical reinforcement learning (2002, methodology): “The G-Algorithm (Chapman and Kaelbling, 1991) and U Tree (McCallum, 1995) both apply propositional decision tree techniques to reinforcement learning.”
Predictive Representations of State (2001, background): “McCallum (1995) has shown in a number of examples that sophisticated history-based methods can be e ective in large problems, and are often more practical than POMDP methods even in small ones.”
Hierarchical Memory-Based Reinforcement Learning (2000, unknown): “Short-term memory consisting of a linear (or tree-based) sequence of primitive actions and observations has been shown to be a useful strategy [2].”
Tree-Based Batch Mode Reinforcement Learning (2005, unknown): “Several authors who adapted regression trees in other ways to reinforcement learning have suggested the use of other score criteria for example based on the violation of the Markov assumption (McCallum, 1996; Uther and Veloso, 1998) or on the combination of several error terms like the supervised, t”
Policy-Gradient Algorithms for Partially Observable Markov Decision Processes (2003, methodology): “To introduce POMDPs let us consider an example based on the work of McCallum [1996], where an agent learns to drive a car in New York.”
An algebraic approach to abstraction in reinforcement learning (2004, methodology): “refine the partition with more learning or planning (e.g., Whitehead and Ballard, 1991; McCallum, 1995; Jonsson and Barto, 2001; Kim and Dean, 2001; Feng et al., 2003).”
An Introduction to Deep Reinforcement Learning (2018, unknown): “However, increasing the size of the set of candidate optimal policies generally implies: (i) more computation to search within this set (Singh et al., 1994; McCallum, 1996) and, (ii) an increased risk of including candidate policies suffering from overfitting”
Reinforcement learning for factored Markov decision processes (2002, methodology): “I show that this DBN algorithm can solve POMDPs with very large state spaces and useful hidden state.”
Multi-task Reinforcement Learning in Partially Observable Stochastic Environments (2009, unknown): “To make notation simple, the following conventions are observed throughout the paper:”
Bayesian Reinforcement Learning in Factored POMDPs (2018, methodology): “More traditional approaches including the U-Tree algorithm [29] (and its modifications), EM-based algorithms such as [25] and policy gradient methods [4], typically do not suffer from the same lack of sample efficiency.”