Learning Dynamic Robot-to-Human Object Handover from Human Feedback
By Andras Gabor Kupcsik, David Hsu, Wee Sun Lee. 2016
92 citations recorded by the source corpus.
Object handover is a basic, but essential capability for robots interacting with humans in many applications, e.g., caring for the elderly and assisting workers in manufacturing workshops. It appears deceptively simple, as humans perform object handover almost flawlessly. The success of humans, however, belies the complexity of object handover as collaborative physical interaction between two agents with limited communication. This paper presents a learning algorithm for dynamic object handover, for example, when a robot hands over water bottles to marathon runners passing by the water station. We formulate the problem as contextual policy search, in which the robot learns object handover by interacting with the human. A key challenge here is to learn the latent reward of the handover task under noisy human feedback. Preliminary experiments show that the robot learns to hand over a water bottle naturally and that it adapts to the dynamics of human motion. One challenge for the future is to combine the model-free learning algorithm with a model-based planning approach and enable the robot to adapt over human preferences and object characteristics, such as shape, weight, and surface texture.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Yoshua Bengio
- Trevor Darrell
- Ruslan Salakhutdinov
- Percy Liang
- Stefano Ermon
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Ming-Hsuan Yang
- Li Fei-Fei
- Jitendra Malik
- Xiaogang Wang
- Yi Yang
In Control and Systems Engineering
- Sergey Levine
- Gerhard Neumann
- Pieter Abbeel
- Chelsea Finn
- Jan Peters
- Stefano Ermon
- Rafael Rafailov
- Takayuki Osa
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Gerhard Neumann
- Chelsea Finn
- Jan Peters
- Aldo Pacchiano
- Shimon Whiteson
- Jonathan Lee
Citations in the corpus
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023, background): “the reward model) and subsequently optimizing it [15, 9, 12, 31, 17].”
- Dueling RL: Reinforcement Learning with Trajectory Preferences (2021, methodology): “This framework widely applies to multiple areas including robot training, stock-prediction, recommender systems, clinical trials, etc. (Novoseller et al., 2019; Sadigh et al., 2017; Christiano et al., 2017; Kupcsik et al., 2018; Jain et al., 2013; Wirth et al., 2017).”
- Dueling Posterior Sampling for Preference-Based Reinforcement Learning (2019, methodology): “, 2017) to human-robot interaction (Kupcsik et al., 2018), and take”
- Sample and Feedback Efficient Hierarchical Reinforcement Learning from Human Preferences (2018, methodology): “We compare our algorithm to two baselines, where i) the left policy is used for all rollouts (K = 1) making the use of a hierarchical architecture obsolete, and ii) the ORM is omitted and the CRMs are directly learned from preferences (without ORM) similar to [19].”
- Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons (2023, background): “…recommender systems, clinical trials, large language models etc. (Novoseller et al., 2019; Sadigh et al., 2017; Christiano et al., 2017b; Kupcsik et al., 2018; Jain et al., 2013; Wirth et al., 2017; Knox and Stone, 2008; MacGlashan et al., 2017; Christiano et al., 2017a; Warnell et al.,…”
- A Survey of Preference-Based Reinforcement Learning Methods (2017, methodology): “Kupcsik et al. (2015) introduced an approach that can use preference as well as numeric feedback.”