Stable reinforcement learning with autoencoders for tactile and visual data

By Herke Van Hoof, Nutan Chen, Maximilian Karl, Patrick van der Smagt, Jan Peters. 2016

185 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biomedical Engineering, Cognitive Neuroscience, Reinforcement Learning, Representation & Self-Supervised Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Cognitive Neuroscience

In Reinforcement Learning

In Representation & Self-Supervised Learning

In Robotics & Control

Citations in the corpus

Unsupervised state representation learning with robotic priors: a robustness benchmark (2017, methodology): “…bears some similarity with approaches such as Embed to Control [12], which learns states that follow a linear dynamic, or the approach of [11], [13], [14] which learn states that make possible to reconstruct the next observation with models such as PSRs (predictive state representation)…”
Variational Autoencoders Pursue PCA Directions (by Accident) (2018, background): “a representation is found it allows for interpretable latent code manipulation, which is desirable in a variety of applications; recently, for example, in reinforcement learning [39, 18, 11, 41, 34].”
Continual Learning for Robotics (2019, background): “We can still note several approaches that successfully tackle this problem, either in an end-to-end manner [67, 121], or by splitting the two challenges to address them separately, i.e., by ﬁrst learning a state representation [82] and later performing policy learning [45, 164, 100, 2, 38, 64].”
Deep predictive policy training using reinforcement learning (2017, methodology): “However, in recent years a considerable amount of research is devoted to learning state representations from raw observation data (e.g., [6]–[11]).”
Learning robot in-hand manipulation with tactile features (2015, background): “We believe this effect is due to the effect of noise in high-dimensional input spaces [9].”
Taming VAEs (2018, background): “Deep variational auto-encoders (VAEs) have made substantial progress in the last few years on modelling complex data such as images [1, 2, 3, 4, 5, 6, 7], speech synthesis [8, 9], molecular discovery [10, 11, 12], robotics [13, 14, 15] and 3d-scene understanding [16, 17, 18].”
SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning (2018, methodology): “(d) Our method ﬁnds an embedding of observed trajectories into a latent space where local linear models produce a better ﬁt. for dynamical systems is typically aimed at improving reconstruction or forward prediction [18, 2, 41, 43], which largely beneﬁts model-free RL and global model methods.”
A Review of Tactile Information: Perception and Action Through Touch (2020, methodology): “For this task, some work focused on the computation of contact-and object-level information [191], [193], [167], [200], but the majority of work wastostudythetactilecontrollerstorelocatethegraspedobjects.”
Electronic skins and machine learning for intelligent soft robots (2020, methodology): “(86) used autoencoders to generate a low-dimensional representation of their complex and continuous tactile data.”
Learning Actionable Representations from Visual Observations (2018, methodology): “[15] use variational auto-encoders to stabilize a reinforcement learning system based on visual and tactile data streams.”
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning (2018, unknown): “nds an embedding of observed trajectories into a latent space where local linear models produce a better ﬁt. for dynamical systems is typically aimed at improving reconstruction or forward prediction [18, 2, 41, 43], which largely beneﬁts model-free RL and global model methods. Instead, in this work”
Explainability in Deep Reinforcement Learning (2020, background): “Other works that facilitate the explainability of RL models by using reward-based losses for more interpretable RL are in [65, 49, 39]. ﬁre-main-engine) for HRA in Lunar Lander before a crash.”
Manipulation by Feel: Touch-Based Control with Deep Predictive Models (2019, methodology): “Prior work has also used reinforcement learning to learn to stabilize an object with touch sensing [18] and explored learning forward predictive models for touch sensors [19].”
S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay (2019, unknown): “They are often used for SRL [18], [7], along with other types of auto-encoders, because the reconstruction error provide an intuitive way of analyzing what the model has learned in the latent representation.”