Memory Approaches to Reinforcement Learning in Non-Markovian Domains

By Longxin Lin, Tom Michael Mitchell. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Reinforcement Learning with Unsupervised Auxiliary Tasks (2016, background): “Although learning environment models as auxiliary tasks could improve RL agents (e.g. Lin & Mitchell (1992); Li et al. (2015)), this has not yet been shown to work in rich visual environments.”
Causally Correct Partial Models for Reinforcement Learning (2020, neutral): “…next-step models (Oh et al., 2015; Ha & Schmidhuber, 2018; Chiappa et al., 2017; Schmidhuber, 2010; Xie et al., 2016; Deisenroth & Rasmussen, 2011; Lin & Mitchell, 1992; Li et al., 2015; Diuk et al., 2008; Igl et al., 2018; Ebert et al., 2018; Kaiser et al., 2019; Janner et al., 2019).”
Shaping Belief States with Generative Environment Models for RL (2019, background): “The idea of learning general world models to support decision-making is probably one of the most pervasive ideas in AI research, [30, 11, 41, 42, 14, 43, 44, 45, 46, 33, 30, 47, 48, 3, 49, 50, 51, 52].”
Reinforcement Learning: A Survey (1996, methodology): “Lin and Mitchell (1992) used a (cid:12)xed-width (cid:12)nite history window to learn a pole balancing task.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “1 C as a Standard RL Machine whose States are M ’s Activations We start with details of an approach whose principles date back to the early 1990s [227, 150] (Sec.”
Contextual Behaviors and Internal Representations Acquired by Reinforcement Learning with a Recurrent Neural Network in a Continuous State and Action Space Task (2008, methodology): “After the proposition of the “Recurrent-Q” technique[1], several works have followed[2][3].”
On Q-learning Convergence for Non-Markov Decision Processes (2018, methodology): “…approach to handle the non-Markovian nature of a POMDP is to estimate a Markovian model with a state estimation method [Whitehead and Lin, 1995; Lin and Mitchell, 1992; Cassandra et al. , 1994; Cassandra, 1994] or use a ﬁnite subset of the recent history as a state [McCallum, 1995] to form a k…”
CAQL: Continuous Action Q-Learning (2019, methodology): “Typically, the loss is minimized over mini-batches B of past data ( x, a, r, x (cid:48) ) sampled from a large experience replay buffer R (Lin & Mitchell, 1992).”
Algorithms for Sequential Decision Making (1996, background): “This approach has been studied by a number of researchers [106, 89, 137].”
Accelerated Neural Evolution through Cooperatively Coevolved Synapses (2008, methodology): “success in tasks such as pole-balancing (Lin and Mitchell, 1992), pursuit-evasion games (Lin, 1992), and backgammon (Tesauro, 1992).”
Real-time reinforcement learning by sequential Actor-Critics and experience replay (2009, methodology): “This idea of repeating many similar operations to the same event, called experience replay (Mahadevan & Connell, 1992; Lin, 1992; Lin & Mitchell, 1992; Cichosz, 1999), was popular a few years ago but has received little attention recently.”
A POMDP formulation of preference elicitation problems (2002, background): “More general approximators, such as feedforward neural networks, seem more promising [12].”
RecSim: A Configurable Simulation Platform for Recommender Systems (2019, unknown): “We do not focus on modules speciﬁc to RL algorithms (e.g., replay memory [Lin and Mitchell, 1992]).”
Policy-Gradient Algorithms for Partially Observable Markov Decision Processes (2003, methodology): “further reviewed by Lippmann [1989], Haton [1997], and Schuster [1999].”