Unsupervised state representation learning with robotic priors: a robustness benchmark

By Timothée Lesort, Mathieu Seurin, Xinrui Li, Natalia Díaz Rodríguez, David Filliat. 2017

33 citations recorded by the source corpus.

Our understanding of the world depends highly on our capacity to produce intuitive and simplified representations which can be easily used to solve problems. We reproduce this simplification process using a neural network to build a low dimensional state representation of the world from images acquired by a robot. As in Jonschkowski et al. 2015, we learn in an unsupervised way using prior knowledge about the world as loss functions called robotic priors and extend this approach to high dimension richer images to learn a 3D representation of the hand position of a robot from RGB images. We propose a quantitative evaluation of the learned representation using nearest neighbors in the state space that allows to assess its quality and show both the potential and limitations of robotic priors in realistic environments. We augment image size, add distractors and domain randomization, all crucial components to achieve transfer learning to real robots. Finally, we also contribute a new prior to improve the robustness of the representation. The applications of such low dimensional state representation range from easing reinforcement learning (RL) and knowledge transfer across tasks, to facilitating learning from raw data with more efficient and compact high level representations. The results show that the robotic prior approach is able to extract high level representation as the 3D position of an arm and organize it into a compact and coherent space of states in a challenging dataset.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Control and Systems Engineering, Representation & Self-Supervised Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Control and Systems Engineering

In Representation & Self-Supervised Learning

In Robotics & Control

Citations in the corpus

State Representation Learning for Control: An Overview (2018, methodology): “This constraint is called causality prior in [Jonschkowski and Brock, 2015] and [Lesort et al., 2017].”
Unsupervised Visuomotor Control through Distributional Planning Networks (2019, background): “Related to learning goal representations, a number of prior works have considered the problem of learning control-centric state representations from interaction data [38, 5, 22, 30, 45, 29], for use with planning or reinforcement learning”
PERCEPTUAL GOAL SPECIFICATIONS FOR REINFORCEMENT LEARNING (2017, background): “Many recent works have trained agents such as simulated and real robots from the raw pixels of videos of humans [13, 37, 38, 48].”
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics (2018, background): “However, since the learning process remains time consuming and the availability of robots is limited by cost and maintenance constraints, several approaches prefer to iterate at first in simulation to learn robotics tasks (Jonschkowski & Brock, 2015; Watter et al., 2015; Curran et al., 2016; Lesort ”
S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning (2018, methodology): “assessed visually, KNN-MSE (Lesort et al., 2017) derives a quantitative metric from this information.”
Learning Actionable Representations from Visual Observations (2018, background): “[13] introduce the reference point prior in their work in addition to the priors introduced in [4].”