Tensor Based Knowledge Transfer Across Skill Categories for Robot Control

By Chenyang Zhao, Timothy M. Hospedales, Freek Stulp, Olivier Sigaud. 2017

25 citations recorded by the source corpus.

Advances in hardware and learning for control are enabling robots to perform increasingly dextrous and dynamic control tasks. These skills typically require a prohibitive amount of exploration for reinforcement learning, and so are commonly achieved by imitation learning from manual demonstration. The costly non-scalable nature of manual demonstration has motivated work into skill generalisation, e.g., through contextual policies and options. Despite good results, existing work along these lines is limited to generalising across variants of one skill such as throwing an object to different locations. In this paper we go significantly further and investigate generalisation across qualitatively different classes of control skills. In particular, we introduce a class of neural network controllers that can realise four distinct skill classes: reaching, object throwing, casting, and ball-in-cup. By factorising the weights of the neural network, we are able to extract transferrable latent skills, that enable dramatic acceleration of learning in cross-task transfer. With a suitable curriculum, this allows us to learn challenging dextrous control tasks like ball-in-cup from scratch with pure reinforcement learning.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgetting (2020, methodology): “Meta-RL [9, 11, 15, 8] and multi-task RL [27, 36, 40, 43] also seek to accelerate learning by reusing information from different tasks, but in those settings the agent does not handle tasks arriving sequentially and the consequent problem of catastrophic forgetting.”
Continual Learning for Robotics (2019, background): “Transfer learning Transfer learning (Pratt, 1993; Finn et al., 2017; Zhao et al., 2017) is the ability to use what has been learned from a previous task on a new task.”
Learning to Learn: Meta-Critic Networks for Sample Efficient Learning (2017, methodology): “Few-shot learning There is extensive work on meta-learning to enable few-shot supervised [37, 27, 18, 20, 4] and reinforcement [8, 34, 39, 30] learning.”
DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics (2020, methodology): “The transfer procedure is illustrated in Figure 23, with full details available in [87].”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “Zhao et al. (2017) followed a similar dictionary-learning formulation for deep networks in the batch MTL setting, replacing all matrix operations with equivalent tensor operations.”
Understanding Continual Learning Settings with Data Distribution Drift Analysis (2021, background): “…and techniques from other research domains such as few-shot learning (Lake et al., 2011; Fei-Fei et al., 2006), transfer learning (Pratt, 1993; Zhao et al., 2017), knowledge distillation (Hinton et al., 2015), meta-learning (Brazdil et al., 2008; Hospedales et al., 2020), learning from sparse…”