Controlling Assistive Robots with Learned Latent Actions

By Dylan P. Losey, Krishnan Srinivasan, Ajay Mandlekar, Animesh Garg, Dorsa Sadigh. 2019

79 citations recorded by the source corpus.

Assistive robotic arms enable users with physical disabilities to perform everyday tasks without relying on a caregiver. Unfortunately, the very dexterity that makes these arms useful also makes them challenging to teleoperate: the robot has more degrees-of-freedom than the human can directly coordinate with a handheld joystick. Our insight is that we can make assistive robots easier for humans to control by leveraging latent actions. Latent actions provide a lowdimensional embedding of high-dimensional robot behavior: for example, one latent dimension might guide the assistive arm along a pouring motion. In this paper, we design a teleoperation algorithm for assistive robots that learns latent actions from task demonstrations. We formulate the controllability, consistency, and scaling properties that user-friendly latent actions should have, and evaluate how different lowdimensional embeddings capture these properties. Finally, we conduct two user studies on a robotic arm to compare our latent action approach to both state-of-the-art shared autonomy baselines and a teleoperation strategy currently used by assistive arms. Participants completed assistive eating and cooking tasks more efficiently when leveraging our latent actions, and also subjectively reported that latent actions made the task easier to perform. The video accompanying this paper can be found at: https://youtu.be/wjnhrzugBj4.

Subfields: Artificial Intelligence, Control and Systems Engineering, Social Psychology, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Social Psychology

In Reinforcement Learning

Citations in the corpus

LILA: Language-Informed Latent Actions (2021, methodology): “We implement the encoder and decoder as multi-layer feed-forward networks, with the ReLU activation as in prior work [7, 28].”
Shared Autonomy with Learned Latent Actions (2020, methodology): “For example, the robot can use demonstrations to learn an intelligent mapping between the human’s low-dimensional inputs and the robot’s high-dimensional actions [19].”
Learning latent actions to control assistive robots (2021, unknown): “This paper combines our earlier work from [31,26,30].”
Learning Visually Guided Latent Actions for Assistive Teleoperation (2021, methodology): “Learned latent actions (Losey et al., 2020) tackles this issue by using conditional autoencoders (CAEs) (Doersch, 2016) to map from low-dimensional inputs to high-dimensional actions, providing users with intuitive, task-conditional control.”
Learning User-Preferred Mappings for Intuitive Robot Control (2020, methodology): “The teleoperation controller in our experiments follows the structure in [21], and the human alignment model is constructed as in Section IV-A.”