Towards Knowledgeable Supervised Lifelong Learning Systems

By Diana Benavides Prado, Yun Sing Koh, Patricia J. Riddle. 2020

11 citations recorded by the source corpus.

Learning a sequence of tasks is a long-standing challenge in machine learning. This setting applies to learning systems that observe examples of a range of tasks at different points in time. A learning system should become more knowledgeable as more related tasks are learned. Although the problem of learning sequentially was acknowledged for the first time decades ago, the research in this area has been rather limited. Research in transfer learning, multitask learning, metalearning and deep learning has studied some challenges of these kinds of systems. Recent research in lifelong machine learning and continual learning has revived interest in this problem. We propose Proficiente, a full framework for long-term learning systems. Proficiente relies on knowledge transferred between hypotheses learned with Support Vector Machines. The first component of the framework is focused on transferring forward selectively from a set of existing hypotheses or functions representing knowledge acquired during previous tasks to a new target task. A second component of Proficiente is focused on transferring backward, a novel ability of long-term learning systems that aim to exploit knowledge derived from recent tasks to encourage refinement of existing knowledge. We propose a method that transfers selectively from a task learned recently to existing hypotheses representing previous tasks. The method encourages retention of existing knowledge whilst refining. We analyse the theoretical properties of the proposed framework. Proficiente is accompanied by an agnostic metric that can be used to determine if a long-term learning system is becoming more knowledgeable. We evaluate Proficiente in both synthetic and real-world datasets, and demonstrate scenarios where knowledgeable supervised learning systems can be achieved by means of transfer.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, background): “Early techniques under lifelong learning mainly perform knowledge transfer but do not tackle CF (Chen and Liu 2014; Ruvolo and Eaton 2013; Benavides-Prado, Koh, and Riddle 2020).”
Continual Learning by Using Information of Each Class Holistically (2021, background): “Traditional lifelong learning methods are also related (Ru-volo and Eaton 2013; Chen and Liu 2014; Benavides-Prado, Koh, and Riddle 2020).”
A Theory for Knowledge Transfer in Continual Learning (2022, methodology): “Although novel, these bounds were speciﬁc to the implementation using SVM.”