Continual Learning for Robotics

By Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, Natalia Díaz Rodríguez. 2019

310 citations recorded by the source corpus.

Continual learning (CL) is a particular machine learning paradigm where the data distribution and learning objective changes through time, or where all the training data and objective criteria are never available at once. The evolution of the learning process is modeled by a sequence of learning experiences where the goal is to be able to learn new skills all along the sequence without forgetting what has been previously learned. Continual learning also aims at the same time at optimizing the memory, the computation power and the speed during the learning process. An important challenge for machine learning is not necessarily finding solutions that work in the real world but rather finding stable algorithms that can learn in real world. Hence, the ideal approach would be tackling the real world in a embodied platform: an autonomous agent. Continual learning would then be effective in an autonomous agent or robot, which would learn autonomously through time about the external world, and incrementally develop a set of complex skills and knowledge. Robotic agents have to learn to adapt and interact with their environment using a continuous stream of observations. Some recent approaches aim at tackling continual learning for robotics, but most recent papers on continual learning only experiment approaches in simulation or with static datasets. Unfortunately, the evaluation of those algorithms does not provide insights on whether their solutions may help continual learning in the context of robotics. This paper aims at reviewing the existing state of the art of continual learning, summarizing existing benchmarks and metrics, and proposing a framework for presenting and evaluating both robotics and non robotics approaches in a way that makes transfer between both fields easier.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Infectious Diseases, Meta & Continual Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Infectious Diseases

In Meta & Continual Learning

In Robotics & Control

Citations in the corpus

DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, methodology): “Continual Learning: Continual learning (CL) is the ability of a model to learn new skills without forgetting previous knowledge [Lesort et al., 2019].”
Continual Pre-Training Mitigates Forgetting in Language and Vision (2022, background): “Continual Learning (CL) (Lesort et al., 2020) focuses on the design of agents able to learn from a stream of non-stationary data while preserving previously acquired knowledge.”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “In this section, we introduce the formalization of continual learning, following the recent works in this area [13, 18, 19].”
RECALL: Replay-based Continual Learning in Semantic Segmentation (2021, background): “Current techniques can be grouped into four main (non mutually exclusive) categories [24]: namely, dynamic architectures, regularization-based, rehearsal and generative replays.”
Self-Supervised Training Enhances Online Continual Learning (2021, background): “, continual object detection [1, 62], continual semantic segmentation [11, 48], or continual learning for robotics applications [23, 37].”
On Class Orderings for Incremental Learning (2020, background): “In recent years, many methods have been proposed to alleviate this problem which stands in the way to advanced life-long learning systems (Lesort et al., 2020; De Lange et al., 2019; Parisi et al., 2019).”
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning (2020, background): “Continual Learning studies the problem of learning from a data stream with changing data distributions over time [20, 23].”
Continual learning: A comparative study on how to defy forgetting in classification tasks (2019, background): “Continual learning has been the subject of several recent surveys [9], [25], [26], [27].”
Latent Replay for Real-Time Continual Learning (2019, background): “Recent surveys [19, 10] provide an overview of the CL field.”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, methodology): “Thus, task-agnostic (or task-free) CL has been motivated and studied in recent works (Aljundi et al., 2019b; Zeno et al., 2018; He et al., 2019; Lesort et al., 2019b).”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “Accordingly, recently developed experimental protocols in deep continual learning (Farquhar and Gal, 2018b; Kemker et al., 2018; Parisi et al., 2019; De Lange et al., 2021; Lesort et al., 2019; Pfülb and Gepperth, 2019; Lesort et al., 2020) seem to mainly occupy themselves with evaluation procedures”
Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches (2019, background): “However, this approach is rather impractical when learning from high-dimensional streaming data, especially in highly constrained computational platforms and embedded systems [15, 34].”
Defining Benchmarks for Continual Few-Shot Learning (2020, methodology): “The problem of continual learning (CL), also called lifelong learning, has been considered since the beginnings of artificial intelligence and it remains an open challenge in robotics (Lesort et al., 2019c) and machine learning (Parisi et al., 2019).”
Regularization Shortcomings for Continual Learning (2019, methodology): “We direct the reader to surveys on the topics for a more in-depth state of the art [21, 16].”
CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions (2020, background): “AI also known as Incremental or Lifelong Learning, has received a renewed attention from the research community over the last few years (Parisi et al., 2019; Lomonaco, 2019; Lesort et al., 2020).”