Shared Autonomy with Learned Latent Actions

By Hong Jun Jeon, Dylan P. Losey, Dorsa Sadigh. 2020

91 citations recorded by the source corpus.

Assistive robots enable people with disabilities to conduct everyday tasks on their own. However, these tasks can be complex, containing both coarse reaching motions and fine-grained manipulation. For example, when eating, not only does one need to move to the correct food item, but they must also precisely manipulate the food in different ways (e.g., cutting, stabbing, scooping). Shared autonomy methods make robot teleoperation safer and more precise by arbitrating user inputs with robot controls. However, these works have focused mainly on the high-level task of reaching a goal from a discrete set, while largely ignoring manipulation of objects at that goal. Meanwhile, dimensionality reduction techniques for teleoperation map useful high-dimensional robot actions into an intuitive low-dimensional controller, but it is unclear if these methods can achieve the requisite precision for tasks like eating. Our insight is that---by combining intuitive embeddings from learned latent actions with robotic assistance from shared autonomy---we can enable precise assistive manipulation. In this work, we adopt learned latent actions for shared autonomy by proposing a new model structure that changes the meaning of the human's input based on the robot's confidence of the goal. We show convergence bounds on the robot's distance to the most likely goal, and develop a training procedure to learn a controller that is able to move between goals even in the presence of shared autonomy. We evaluate our method in simulations and an eating user study. See videos of our experiments here: this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

LILA: Language-Informed Latent Actions (2021, methodology): “Instead, we adopt learned latent actions [5, 7, 8, 27, 28] a framework that uses conditional auto-encoders [29] to learn task-speciﬁc latent “action” spaces from demonstrations.”
Learning with Humans in the Loop (2020, background): “to scale to more complex shared autonomy scenarios [31].”
Learning Visually Guided Latent Actions for Assistive Teleoperation (2021, methodology): “Prior work tackles this by learning an embedding function that maps low-dimensional human inputs to high-dimensional robot actions (Losey et al., 2020; Jeon et al., 2020).”
Learning latent actions to control assistive robots (2021, unknown): “This paper combines our earlier work from [31,26,30].”