Learning on the Job: Online Lifelong and Continual Learning

By Bing Liu. 2020

55 citations recorded by the source corpus.

One of the hallmarks of the human intelligence is the ability to learn continuously, accumulate the knowledge learned in the past and use the knowledge to help learn more and learn better. It is hard to imagine a truly intelligent system without this capability. This type of learning differs significantly than the classic machine learning (ML) paradigm of isolated single-task learning. Although there is already research on learning a sequence of tasks incrementally under the names of lifelong learning or continual learning, they still follow the traditional two-phase separate training and testing paradigm in learning each task. The tasks are also given by the user. This paper adds on-the-job learning to the mix to emphasize the need to learn during application (thus online) after the model has been deployed, which traditional ML cannot do. It aims to leverage the learned knowledge to discover new tasks, interact with humans and the environment, make inferences, and incrementally learn the new tasks on the fly during applications in a self-supervised and interactive manner. This is analogous to human on-the-job learning after formal training. We use chatbots and self-driving cars as examples to discuss the need, some initial work, and key challenges and opportunities in building this capability.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Lifelong and Continual Learning Dialogue Systems: Learning during Conversation (2021, methodology): “Definition (Lifelong and Continual Learning): It consists of two parts (Liu 2020):”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “[150] introduces on-the-job learning, which requires a deployed model to discover new tasks, collect training data continuously and incrementally learn new tasks without interrupting the application.”
Knowledge Capture and Replay for Continual Learning (2020, methodology): “of these models, it remains difficult to extract and reuse the knowledge embedded in the representations learned by these models in a benign manner for similar or other downstream tasks [Bing, 2020].”
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning (2020, background): “This setting reflects many real-world data streams [6, 25] and offers a challenging testbed for online CL research.”
Self-Initiated Open World Learning for Autonomous AI Agents (2021, background): “Classic machine learning (ML) makes closed world assumption, which means that what are seen by the system in testing or deployment must have been seen in training (Fei and Liu 2016; Bendale and Boult 2015; Liu 2020), i.”
Lifelong Learning Dialogue Systems: Chatbots that Self-Learn On the Job (2020, background): “, after model deployment and during conversing or interacting with the (human) end users and thereby, improve its capability over time in a self-supervised manner (Chen and Liu 2018; Liu 2020).”
AI Autonomy: Self-initiated Open-world Continual Learning and Adaptation (2022, background): “…makes the closed-world assumption, which means that the classes of objects seen by the system in testing or application must have been seen during training (Fei, Wang, and Liu 2016; Bendale and Boult 2015; Liu 2020), i.e., there is nothing unexpected or novel occurring in testing or deployment.”
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning (2023, background): “Open world learning has been studied by many researchers [54, 53, 6, 15, 37, 38].”