Unsupervised Visuomotor Control through Distributional Planning Networks

By Tianhe Yu, Gleb Shevchuk, Dorsa Sadigh, Chelsea Finn. 2019

42 citations recorded by the source corpus.

While reinforcement learning (RL) has the potential to enable robots to autonomously acquire a wide range of skills, in practice, RL usually requires manual, per-task engineering of reward functions, especially in real world settings where aspects of the environment needed to compute progress are not directly accessible. To enable robots to autonomously learn skills, we instead consider the problem of reinforcement learning without access to rewards. We aim to learn an unsupervised embedding space under which the robot can measure progress towards a goal for itself. Our approach explicitly optimizes for a metric space under which action sequences that reach a particular state are optimal when the goal is the final state reached. This enables learning effective and control-centric representations that lead to more autonomous reinforcement learning algorithms. Our experiments on three simulated environments and two real-world manipulation problems show that our method can learn effective goal metrics from unlabeled interaction, and use the learned goal metrics for autonomous reinforcement learning.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

RoboNet: Large-Scale Multi-Robot Learning (2019, methodology): “8k), Fetch (5k) [6], GoogleRobot (56k) [5]”
Reinforcement Learning without Ground-Truth State (2019, methodology): “Examples of this include using another robot arm to hold all relevant objects [2], placing an IMU sensor [3, 4] or motion capture markers on such objects [5], or ensuring that all relevant objects are placed on scales [6].”
Learning Visually Guided Latent Actions for Assistive Teleoperation (2021, background): “Additional work has shown progress in learning visual representations end-to-end using task reward signals (Lai et al., 2012; Levine et al., 2016; Zhu et al., 2017; Yu et al., 2019), or through self-supervised encoding of play behavior (Wang et al.”
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation (2019, background): “In practice, acquiring cost functions that accurately reﬂect the distance between two images is a challenging problem (Yu et al., 2019).”