An on-line algorithm for dynamic reinforcement learning and planning in reactive environments

By J. Urgen Schmidhuber. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

From Pixels to Torques: Policy Learning with Deep Dynamical Models (2015, methodology): “o learn forward models of the underlying dynamical system, which are then used for internal simulations and policy learning. These ideas have been successfully applied to RL, control and robotics in (Schmidhuber, 1990; Atkeson & Schaal, 1997; Bagnell & Schneider, 2001; Contardo et al., 2013;”
Recurrent World Models Facilitate Policy Evolution (2018, methodology): “Many basic concepts ﬁrst explored in the 1980s for feed-forward neural networks (FNNs) [56, 58, 72, 104, 105] and in the 1990s for RNNs [74, 75, 76, 78] laid some of the groundwork for Learning to Think [83].”
Value Iteration Networks (2016, methodology): “agation. This makes our policy simple to train using standard RL and IL algorithms, and straightforward to integrate with NNs for perception and control. Our approach is different from model-based RL [22, 5], which requires system identiﬁcation to map the observations to a dynamics model, which is t”
Deep learning in neural networks: An overview (2014, methodology): “While the RNN methods above were designed for DL of temporal sequences, the Neural Heat Exchanger (Schmidhuber, 1990c) consists of two parallel deep FNNs with opposite flow directions.”
COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration (2019, positive): “This approach is a form of curiosity-driven exploration, as previously described in both the psychology (Gopnik et al., 1999) and the reinforcement learning literature (Pathak et al., 2017; Schmidhuber, 1990a,b).”
World Models (2018, methodology): “Using RNNs to develop internal models to reason about the future has been explored as early as 1990 in a paper called Making the World Differentiable (Schmidhuber, 1990a), and then further explored in (Schmidhuber, 1990b; 1991a; 1990c).”
Imagination-Augmented Agents for Deep Reinforcement Learning (2017, methodology): “The general idea of learning to leverage an internal model in arbitrary ways was also discussed by Schmidhuber [44].”
Multi-task Deep Reinforcement Learning with PopArt (2018, background): “…reinforcement learning that have been explored in the literature: off-policy learning of many predictions about the same stream of experience (Schmidhuber 1990; Sutton et al. 2011; Jaderberg et al. 2016), continual learning in a sequence of tasks (Ring 1994; Thrun 1996; Thrun 2012; Rusu et…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “M has m+n+o input units to receive all(t) at time t < tdeath, and m + n output units to produce a prediction pred(t + 1) ∈ R of sense(t + 1) [223, 226, 222, 227].”
Metacontrol for Adaptive Imagination-Based Optimization (2017, methodology): “We also build on classic work by Schmidhuber (1990a;b), which used an RL controller with a recurrent neural network (RNN) world model to evaluate and improve upon candidate controls online.”
Visual Semantic Planning Using Deep Successor Representations (2017, neutral): “, [8, 44, 50]) also estimate a environment model.”
One Big Net For Everything (2018, methodology): “Unlike in the present paper, predictions of expected cumulative rewards are actually essential in traditional reinforcement learning [29, 90, 101, 76] where they are usually limited to the case of scalar rewards (while ONE’s rewards may be vector-valued like in old work of 1990 [55, 56]).”
Artificial curiosity with planning for autonomous perceptual and cognitive development (2011, background): “Further, we examine the critical role that planning capabilities [17], [3], [5] play, due to the non-stationary intrinsic reward signal, in curious agents.”
Making the World Di erentiable: On Using Self-Supervised Fully Recurrent Neural Networks for Dynamic Reinforcement Learning (1990, methodology): “Below we describe the parallel version (an improved version of the systems described in [30], [32], and the February version of this report).”
Model-Based Planning in Discrete Action Spaces (2017, background): “Later works [30, 27, 17] explored the idea of backpropagating through learned, approximate forward models of the environment.”
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) (2010, methodology): “1990: Non-traditional RL (without restrictive Markovian assumptions [72]) based on an adaptive recurrent neural network as a predictive world model [68] is used to maximize the controller’s intrinsic reward, which is proportio nal to the model’s prediction errors [67], [71].”
Maximizing Fun by Creating Data with Easily Reducible Subjective Complexity (2013, methodology): “…without restrictive traditional assumptions (Schmidhuber 1991d)) based on an adaptive recurrent neural network as a predictive data encoder (Schmidhuber 1990b) is used to maximize intrinsic reward for the creator in proportion to the model's prediction errors (1990) (Schmidhuber 1990a, 1991c).”