Reinforcement Learning with Long Short-Term Memory

By Bram Bakker. 2001

349 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Benchmarking Deep Reinforcement Learning for Continuous Control (2016, background): “Since we penalize for excessive controls as well as falling over, during the initial stage of learning, when the robot is not yet able to move forward for a sufﬁcient distance without falling, apparent local optima exist including staying at the origin or diving forward slowly.”
Memory-based control with recurrent neural networks (2015, background): “Several groups [15, 1, 5] have studied the use of model-free algorithms with recurrent networks to solve POMDPs with discrete action spaces.”
Offline Reinforcement Learning as One Big Sequence Modeling Problem (2021, methodology): “…works have studied applying sequence models of various types to represent components in standard RL algorithms, such as policies, value functions, and models (Bakker, 2002; Heess et al., 2015a; Chiappa et al., 2017; Parisotto et al., 2020; Parisotto & Salakhutdinov, 2021; Kumar et al., 2020b).”
Deep learning in neural networks: An overview (2014, methodology): “Use an RNN as a value function mapping arbitrary event histories to values (e.g., Bakker, 2002; Lin, 1993; Schmidhuber, 1990b, 1991c).”
RUDDER: Return Decomposition for Delayed Rewards (2018, background): “This is known as “backpropagation through a model” [22, 27, 28, 42, 33, 4, 5].”
Deep Recurrent Q-Learning for Partially Observable MDPs (2015, methodology): “Computational efficiency of RNNs is an important concern.”
Deep Variational Reinforcement Learning for POMDPs (2018, methodology): “There are several approaches that utilise RNN s in POMDP s (Bakker, 2002; Wierstra et al., 2007; Zhang et al., 2015; Heess et al., 2015), including multi-agent settings (Foerster et al., 2016), learning text-based fantasy games (Narasimhan et al., 2015) or, most recently, applied to Atari…”
Reinforcement Learning as One Big Sequence Modeling Problem (2021, background): “Indeed, a number of prior works have studied applying sequence models of various types to represent components in standard RL algorithms, such as policies, value functions, and models [4, 21, 9, 45, 44, 34].”
Causal Analysis of Agent Behavior for AI Safety (2021, background): “In this T-maze environment, the agent must remember a cue placed at the beginning of a corridor in order to know which direction to go at the end of it (Olton, 1979; Bakker, 2001).”
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks (2019, background): “Moursavian et al. [31] use an LSTM policy for target driven navigation.”
Co-evolving recurrent neurons learn deep memory POMDPs (2005, methodology): “Because RL-LSTM is, to our knowledge, currently the most efficient method for solving this class of problems, our experiments use the same setups and compare our results with H-ESP to those published in [1].”
Free-energy-based reinforcement learning in a partially observable environment (2010, methodology): “The matching T-maze task is an extension of the non-Markovian grid-based T-maze task [7] to investigate the coding and combinatorial usage of task-relevant information.”
Reinforcement learning by backpropagation through an LSTM model/critic (2007, methodology): “Variations of this can be used in an RL context to learn the value function [1] based on a variation of Q-learning (see eq.”
Supervised Sequence Labelling with Recurrent Neural Networks (2012, background): “, 2007; Chen and Chaudhari, 2005), music generation (Eck and Schmidhuber, 2002), reinforcement learning (Bakker, 2002), speech recognition (Graves and Schmidhuber, 2005b; Graves et al.”