Residual Policy Learning for Shared Autonomy
By Charles B. Schaff, Matthew R. Walter. 2020
54 citations recorded by the source corpus.
Shared autonomy provides an effective framework for human-robot collaboration that takes advantage of the complementary strengths of humans and robots to achieve common goals. Many existing approaches to shared autonomy make restrictive assumptions that the goal space, environment dynamics, or human policy are known a priori, or are limited to discrete action spaces, preventing those methods from scaling to complicated real world environments. We propose a model-free, residual policy learning algorithm for shared autonomy that alleviates the need for these assumptions. Our agents are trained to minimally adjust the human's actions such that a set of goal-agnostic constraints are satisfied. We test our method in two continuous control environments: Lunar Lander, a 2D flight control domain, and a 6-DOF quadrotor reaching task. In experiments with human and surrogate pilots, our method significantly improves task performance without any knowledge of the human's goal beyond the constraints. These results highlight the ability of model-free deep reinforcement learning to realize assistive agents suited to continuous control settings with little knowledge of user intent.
Subfields: Artificial Intelligence, Control and Systems Engineering, Social Psychology, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dorsa Sadigh
- Siddharth Karamcheti
- Percy Liang
- Sergey Levine
- Peter Stone
- Roberto Mart'in-Mart'in
- Megha Srivastava
- Mohit Bansal
In Control and Systems Engineering
- Siddharth Karamcheti
- Percy Liang
- Megha Srivastava
- Roberto Mart'in-Mart'in
- Peter Stone
- Dorsa Sadigh
- Rohan Chandra
- Ben Abbatematteo
In Social Psychology
In Reinforcement Learning
- Siddharth Karamcheti
- Percy Liang
- Megha Srivastava
- Dorsa Sadigh
- Roberto Mart'in-Mart'in
- Peter Stone
- Rohan Chandra
- Ben Abbatematteo
Citations in the corpus
- LILA: Language-Informed Latent Actions (2021, background): “While other work focuses on “blending” human inputs with possibly task-agnostic policies within the same action space [23, 24], in this work, we focus on assistive teleoperation, where a user is provided a low-dimensional controller (e.g., a joystick, sip-and-puff device) to directly control a…”
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “This approach is particularly appealing in domains where human actions are fairly easy to model, such as shared autonomy, where a human policy can be learned by imitating a set of human actors (183, 182).”