S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning

By Antonin Raffin, Ashley Hill, Kalifou René Traoré, Timothée Lesort, Natalia Díaz Rodríguez, David Filliat. 2018

35 citations recorded by the source corpus.

State representation learning aims at learning compact representations from raw observations in robotics and control applications. Approaches used for this objective are auto-encoders, learning forward models, inverse dynamics or learning using generic priors on the state characteristics. However, the diversity in applications and methods makes the field lack standard evaluation datasets, metrics and tasks. This paper provides a set of environments, data generators, robotic control tasks, metrics and tools to facilitate iterative state representation learning and evaluation in reinforcement learning settings.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, methodology): “The experiments consists of 2D navigation tasks using a 3 wheel omni-directional robot similar to the 2D mobile navigation in [Rafﬁn et al., 2018].”
Acquisition of chess knowledge in AlphaZero (2021, background): “Representation learning in RL agents [45, 46, 47, 48, 49, 50] can be beneficial when used to learn low-dimensional representations for states, policies and actions, hoping to meaningfully capture the variations in agent’s environment and disentangle the contributing factors.”
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies (2018, background): “A common form of self-supervision in active contexts leverages temporal information to predict unseen observations [16, 58, 36, 82], or actions [94, 2, 58, 62].”
S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay (2019, methodology): “For the RL algorithm, we selected the Proximal Policy Optimization (PPO) algorithm [16] of SRL Toolbox [15], which in our case employs an actor-critic network that both predicts the value function and a policy.”
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Active Tasks (2018, background): “lf-supervised approaches [91,56,55, 72,84,52,28,74,13,61]). A common form of selfsupervision in active contexts leverages temporal information to predict unseen observations [16,58,36,82], or actions [94,2,58,62]. Domain adaptive approaches learn features that are task-relevant but domain agnostic [”
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics (2018, methodology): “In order to evaluate the methods, we use four (4) environments proposed in S-RL Toolbox (Rafﬁn et al., 2018) (which uses open-source PyBullet (Coumans et al., 2018)).”
Continual State Representation Learning for Reinforcement Learning using Generative Replay (2018, methodology): “We use the standard version of PPO in the S-RL Toolbox implementation (Raffin et al., 2018), see their code for details.”
Explainability in Deep Reinforcement Learning (2020, methodology): “Hierarchical learning [23] [110] [15] [103] Simultaneous leaning [53] [71] [107] [70] Representation learning [111] [24] [84] [85] [38] [24] [80]”