Learning robot motion control with demonstration and advice-operators

By Brenna D. Argall, Brett Browning, Manuela M. Veloso. 2008

88 citations recorded by the source corpus.

As robots become more commonplace within society, the need for tools to enable non-robotics-experts to develop control algorithms, or policies, will increase. Learning from demonstration (LfD) offers one promising approach, where the robot learns a policy from teacher task executions. Our interests lie with robot motion control policies which map world observations to continuous low-level actions. In this work, we introduce advice-operator policy improvement (A-OPI) as a novel approach for improving policies within LfD. Two distinguishing characteristics of the A-OPI algorithm are data source and continuous state-action space. Within LfD, more example data can improve a policy. In A-OPI, new data is synthesized from a student execution and teacher advice. By contrast, typical demonstration approaches provide the learner with exclusively teacher executions. A-OPI is effective within continuous state-action spaces because high level human advice is translated into continuous-valued corrections on the student execution. This work presents a first implementation of the A-OPI algorithm, validated on a Segway RMP robot performing a spatial positioning task. A-OPI is found to improve task performance, both in success and accuracy. Furthermore, performance is shown to be similar or superior to the typical exclusively teacher demonstrations approach.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Reinforcement learning in robotics: A survey (2013, background): “The instructor may provide additional demonstrations in later learning stages (Latzke et al., 2007; Ross et al., 2011a) and which can also be used as differential feedback (Argall et al., 2008).”
Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner (2012, background): “· At a fixed frequency: In classical imitation learning, the learner uses a demonstration to improve its learning at every policy it performs [1, 2, 11].”
A survey of robot learning from demonstration (2009, unknown): “, 2008], continuous action selection [Argall et al., 2008] and to penalize poorly”
Reinforcement Learning Via Practice and Critique Advice (2010, unknown): “Other work combines LBD and human critiques on behavior (Argall , Browning, and Veloso 2007; 2008) similar to those used in our work, but there is no autonomous practice and no experiments involving end-users.”
Learning and Reproduction of Gestures by Imitation (2010, background): “A large body of work in PbD follows a symbolic approach to the representation and encoding of the tasks, see e.g., [4]–[9].”
Learning From Explanations Using Sentiment and Advice in RL (2017, methodology): “Reference [2] creates policies using demonstrations and advice.”
Socially guided intrinsic motivation for robot learning of motor skills (2013, background): “…and Mataric 2003; Lopes et al. 2011), a scalar reward given by the human teacher (Thomaz 2006), a correction demonstration (Chernova and Veloso 2009), an advice to modulate the wrong behaviour (Argall et al. 2008, 2011), or a label of previously reached goals (Thomaz 2006; Thomaz and Breazeal2008).”