Learning state representation for deep actor-critic control

By Jelle Munk, Jens Kober, Robert Babuška. 2016

70 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Control and Systems Engineering

Citations in the corpus

Unsupervised state representation learning with robotic priors: a robustness benchmark (2017, background): “without using labeled input images. Future work should explore the coupling of the priors with other state learning paradigms, such as learning simple forward and reward models to learn a state space [24], and will tackle more complex tasks, with several moving goals. It also remains to evaluate the”
Loss is its own Reward: Self-Supervision for Reinforcement Learning (2016, background): “ML-DDPG (Munk et al., 2016) extends actor-critic with a one-step predictive model of the successor state and reward.”
State Representation Learning for Control: An Overview (2018, methodology): “• Other evaluation benchmarks tested in the reviewed works in this survey include simulated octopus arms [Engel et al., 2006, Munk et al., 2016], labyrinths [Thomas et al., 2017], navigation grids [Magrans de Abril and Kanai, 2018, Oh et al., 2017], driving cars [Jonschkowski and Brock, 2015], or…”
Many-Goals Reinforcement Learning (2018, methodology): “Munk et al. (2016) looked at the benefit obtained by initializing and freezing the parameters of an agent using parameters obtained by predicting observation and reward at next time step.”
Improving Sample Efficiency in Model-Free Reinforcement Learning from Images (2019, methodology): “…the representation is ﬁrst trained via the au-toencoder, and then either with a policy learned on top of the ﬁxed representation (Lange & Riedmiller, 2010; Munk et al., 2016; Higgins et al., 2017b; Zhang et al., 2018; Nair et al., 2018), or with planning (Mattner et al., 2012; Finn et al., 2015).”
Learning Actionable Representations from Visual Observations (2018, methodology): “[17] propose an approach to map their input to a useful hidden state using ”predictive priors” before training their Actor-Critic model using reinforcement learning.”
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies (2018, methodology): “We compare against several stateof-the-art feature methods, including dynamic modeling [53, 68, 36], curiosity [58], DARLA [33], and ImageNet pretraining [42], enumerated in Figure 10.”
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics (2018, methodology): “Forward models (Munk et al., 2016), inverse models (Shelhamer et al., 2017), data-reconstruction models (Mattner et al., 2012; Curran et al., 2016) or priors knowledge (Jonschkowski & Brock, 2015; Lesort et al., 2017) are several approaches that exploit those environments data to learn meaningful…”
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Active Tasks (2018, methodology): “comparisons by controlling the False Discovery Rate (Q= 20%) with Benjamini-Hochberg [8]. 0 1 2 Blind Visual-Target Navigation 0:0 0:5 1:0 Visual Exploration ImageNet [42] VAE [19] DARLA [33] Forward [53] Inverse [68] Forward Inverse [2] Curiosity [58] Scratch Mid-Level 0:8 1:0 Local Planning Figure”