Incremental Semantically Grounded Learning from Demonstration

By Scott Niekum, Sachin Chitta, Andrew Barto, Bhaskara Marthi, Sarah Osentoski. 2013

135 citations recorded by the source corpus.

Much recent work in robot learning from demonstration has focused on automatically segmenting continuous task demonstrations into simpler, reusable primitives. However, strong assumptions are often made about how these primitives can be sequenced, limiting the potential for data reuse. We introduce a novel method for discovering semantically grounded primitives and incrementally building and improving a finite-state representation of a task in which various contingencies can arise. Specifically, a Beta Process Autoregressive Hidden Markov Model is used to automatically segment demonstrations into motion categories, which are then further subdivided into semantically grounded states in a finite-state automaton. During replay of the task, a data-driven approach is used to collect additional data where they are most needed through interactive corrections, which are then used to improve the finite-state automaton. Together, this allows for intelligent sequencing of primitives to create novel, adaptive behavior that can be incrementally improved as needed. We demonstrate the utility of this technique on a furniture assembly task using the PR2 mobile manipulator.

Subfields: Artificial Intelligence, Control and Systems Engineering, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Industrial and Manufacturing Engineering

Citations in the corpus

Learning World Graphs to Accelerate Hierarchical Reinforcement Learning (2019, neutral): “d G wis more favorable over V rand. 6 Related Works Pivotal state discovery is related to unsupervised sequence segmentation [17, 47, 76, 13, 16] and option or sub-task discovery in the context of RL [48, 6, 72, 30, 56, 53, 52]. Among them, both [52] and [76] employ sequential variational models to ”
DARLA: Improving Zero-Shot Transfer in Reinforcement Learning (2017, background): “…source and target domain representations by utilising observation and reward signals from both domains (Tzeng et al., 2016; Daftry et al., 2016; Parisotto et al., 2015; Guez et al., 2012; Talvitie & Singh, 2007; Niekum et al., 2013; Gupta et al., 2017; Finn et al., 2017; Rajendran et al., 2017).”
CompILE: Compositional Imitation Learning and Execution (2018, methodology): “Our framework is closely related to option discovery (Niekum et al., 2013; Kroemer et al., 2015; Fox et al., 2017; Hausman et al., 2017; Krishnan et al., 2017; Fox et al., 2018), with the main difference being that our inference algorithm is agnostic to what type of option (sub-task) encoding is…”
Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets (2017, unknown): “Examples include [23, 18, 14, 31, 13].”
Compositional Imitation Learning: Explaining and executing one task at a time (2018, methodology): “Our framework is closely related to option discovery [16, 17, 18, 19], with the main difference being that our inference algorithm is agnostic to what type of option (sub-task) encoding is used.”
Tell me Dave: Context-sensitive grounding of natural language to manipulation instructions (2014, background): “Others use demonstrations for learning different behaviors (Niekum et al., 2013).”
Learning Semantic Embedding Spaces for Slicing Vegetables (2019, background): “For example, one may deﬁne the feature vector to include the 3D pose of each object part involved in an assembly task [1], [2].”
Modeling Long-horizon Tasks as Sequential Interaction Landscapes (2020, background): “Another direction is to learn policies for robot-object interaction from demonstrations with the goal to reduce uncertainty with expert demonstrations [64, 30, 44, 43, 42].”
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks (2019, background): “Therefore, furniture assembly is a comprehensive robotics task [6]–[8] requiring reliable 3D perception, high-level planning, and sophisticated control, making it a suitable”
Learning grounded finite-state representations from unstructured demonstrations (2015, methodology): “The resulting finite-state automaton (FSA) can then be used to adaptively replay a complex, multi-step task [Niekum et al., 2013].”
Towards learning hierarchical skills for multi-phase manipulation tasks (2015, background): “While some methods assume that a library of motor primitives already exists and focus on sequencing these primitives [30], [27], other approaches also learn the individual skills by segmenting the human demonstrations [29], [6], [14], [38].”