SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning

By Marvin Zhang, Sharad Vikram, Laura M. Smith, Pieter Abbeel, Matthew J. Johnson, Sergey Levine. 2018

236 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Representation & Self-Supervised Learning

Citations in the corpus

DeepMDP: Learning Continuous Latent Space Models for Representation Learning (2019, methodology): “Although similar deﬁnitions of latent space models have been previously studied (Francois-Lavet et al., 2018; Zhang et al., 2018; Ha & Schmidhuber, 2018; Oh et al., 2017; Hafner et al., 2018; Kaiser et al., 2019; Silver et al., 2017), the parametrizations and training objectives used to learn such…”
Learning Latent Dynamics for Planning from Pixels (2018, background): “The robotics community focuses on video prediction models for planning (Agrawal et al., 2016; Finn & Levine, 2017; Ebert et al., 2018; Zhang et al., 2018) that deal with the visual complexity of the real world and solve tasks with”
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “Many such methods have been known to exhibit excellent performance in conventional off-policy settings, where additional data collection is allowed, but prior data is also utilized (Sutton, 1991; Watter et al., 2015; Zhang et al., 2018; Hafner et al., 2018; Janner et al., 2019).”
Dream to Control: Learning Behaviors by Latent Imagination (2019, background): “SOLAR (Zhang et al., 2019) solves robotic tasks via guided policy search in latent space.”
Mastering Atari with Discrete World Models (2020, background): “Most world model agents with image inputs have thus far been limited to relatively simple control tasks (Watter et al., 2015; Ebert et al., 2017; Ha and Schmidhuber, 2018; Hafner et al., 2018; Zhang et al., 2019; Hafner et al., 2019).”
Planning to Explore via Self-Supervised World Models (2020, background): “We focus on world models that predict ahead in a compact latent space and have recently been shown to solve challenging control tasks from images (Hafner et al., 2019; Zhang et al., 2019).”
Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning (2020, background): “Predictive models for representation learning have been widely used as auxiliary tasks for deep RL (Jaderberg et al., 2017; Oord et al., 2018; Zhang et al., 2019; Aytar et al., 2018; Gregor et al., 2019; Song et al., 2020).”
Learning Actionable Representations with Goal-Conditioned Policies (2018, methodology): “In the context of RL and control, representation learning methods have been used for many down-stream applications (Lesort et al., 2018), including representing value functions (Barreto et al., 2016) and building models (Watter et al., 2015; Assael et al., 2015; Zhang et al., 2018b).”
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning (2021, methodology): “This has been demonstrated by learning dynamics using future predictions (Watter et al., 2015; Oh et al., 2017; Ebert et al., 2017; Banijamali et al., 2018; Ha & Schmidhuber, 2018; Hafner et al., 2018; Ichter & Pavone, 2019; Hafner et al., 2020; Zhang et al., 2019), learning belief representations (”
The Value Equivalence Principle for Model-Based Reinforcement Learning (2020, background): “This is an instantiation of the problem known as representation learning [43, 21, 9, 18, 45, 44, 19, 7].”
A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning (2022, background): “Therefore, model-based reinforcement learning (MBRL) (Janner et al., 2019; Kaiser et al., 2019; Schrittwieser et al., 2020; Zhang et al., 2019b; van Hasselt et al., 2019; Hafner et al., 2019b;a; Lenz et al., 2015), which explicitly builds a predictive model to generate samples for learning RL…”
Deep Hierarchical Planning from Pixels (2022, background): “Director learns a world model that compresses the history of observations into a compact feature space and enables planning in this space (Watter et al., 2015; Zhang et al., 2019).”
Offline Reinforcement Learning from Images with Latent Space Models (2020, background): “, 2018) or in a latent space (Levine et al., 2016; Finn et al., 2016; Watter et al., 2015; Banijamali et al., 2018; Zhang et al., 2019; Hafner et al., 2019, 2020; Ha and Schmidhuber, 2018; Kipf et al., 2019; Chen et al., 2020) and can either learn a policy within the model or deploy shooting-based p”
Deep reinforcement learning: a survey (2020, methodology): “Since accurate forward prediction could be very expensive (Chua et al., 2018; Naga-bandi et al., 2018), SOLAP (Zhang et al., 2019) uses simple models, typically linear models, to provide gradient directions for local policy improvement rather than forward prediction and planning.”