Making the world differentiable: on using self supervised fully recurrent neural networks for dynamic reinforcement learning and planning in non-stationary environments

By J. Urgen Schmidhuber. 1990

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration (2019, positive): “This approach is a form of curiosity-driven exploration, as previously described in both the psychology (Gopnik et al., 1999) and the reinforcement learning literature (Pathak et al., 2017; Schmidhuber, 1990a,b).”
World Models (2018, methodology): “Large RNNs are highly expressive models that can learn rich spatial and temporal representations of data.”
Dream to Control: Learning Behaviors by Latent Imagination (2019, background): “Planning with neural network gradients was shown on small problems (Schmidhuber, 1990; Henaff et al., 2018) but has been challenging to scale (Parmas et al.”
Meta-learning of Sequential Strategies (2019, methodology): “More commonly, traditional approaches have used expectation models [Sutton et al., 2012, Schmidhuber, 1990], or deterministic abstract models [Watter et al., 2015, Silver et al., 2016].”
Predictive Information Accelerates Learning in RL (2020, methodology): “This work falls in another group, where future prediction is used as an auxiliary or representation learning method for model-free RL agents [1, 10, 14, 22, 29, 30, 33, 34, 35, 37].”
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning (2021, methodology): “, 2019) and predicting the future conditioned on past observations (Schmidhuber, 1990; Jaderberg et al., 2017; van den Oord et al., 2018; Shelhamer et al., 2016).”
Learning to Predict Without Looking Ahead: World Models Without Forward Prediction (2019, methodology): “Whether such models are learned from data, or created from domain knowledge, there’s an implicit assumption that an agent’s world model [20, 51, 65] is a forward model for predicting future states.”
Lower Dimensional Kernels for Video Discriminators (2019, methodology): “Schmidhuber 1990, 1991) laid out the fundamental principles underpinning dversarial training of neural networks, as elaborated in Schmiduber (2020).”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “This can be interpreted as D and G playing a mini-max game, as with prior work [193], [194], optimising the value function V (G,D),”
An investigation of model-free planning (2019, background): “…& Werbos, 1998; Wang et al., 2018) has supported the idea that model-based behavior can be learned with general recurrent architectures, with planning computation amortized over multiple discrete steps (Schmidhuber, 1990), but comprehensive demonstrations of its effectiveness are still missing.”