Lifelong Machine Learning

By Zhiyuan Chen, Bing Liu. 2016

530 citations recorded by the source corpus.

Subfields: Information Systems and Management, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

In Reinforcement Learning

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, neutral): “…learning (Lake et al., 2011; Romera-Paredes and Torr, 2015; Vinyals et al., 2016; Pan and Yang, 2010), meta-learning and life-long learning (Thrun and Mitchell, 1995; Silver et al., 2013; Chen and Liu, 2016; Parisi et al., 2018; Golkar et al., 2019; Sodhani et al., 2018; Lopez-Paz et al., 2017).”
Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “For more comprehensive reviews of continual learning, we refer to Chen et al. [11] and Parisi et al. [30]”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “Although there are some early surveys of continual learning with relatively broad coverage [2], [5], [8], important advances in recent years have not been incorporated.”
Knowledge Flow: Improve Upon Your Teachers (2019, background): “…2016), learning without forgetting (Li & Hoiem, 2016), growing a brain (Wang et al., 2017), policy distillation (Rusu et al., 2016a), domain adaptation (Pan & Yang, 2010; Long et al., 2015; Tzeng et al., 2015), knowledge distillation (Hin-ton et al., 2015) or lifelong learning (Chen & Liu, 2016).”
Continual and Multi-Task Architecture Search (2019, methodology): “…of architecture search, where the RNN cell structure evolves when trained on new in-coming data/domains, while maintaining the performance on previously learned data/domains (via our block-sparsity and orthogonality conditions discussed below), thus enabling life-long learning (Chen and Liu, 2016).”
RMM: Reinforced Memory Management for Class-Incremental Learning (2023, background): “Similar tasks are also referred to as continual learning [12, 32] and lifelong learning [3, 9].”
A Theoretical Study on Solving Continual Learning (2022, background): “However, in many applications, there are unknowns in testing, which is called the open world setting [1, 5].”
Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer (2022, background): “Such a learning paradigm is known as continual learning (CL) [5, 23], which has recently attracted much attention.”
Continual Training of Language Models for Few-Shot Learning (2022, background): “In general, overcoming CF is a major goal in CL (Chen and Liu, 2018).”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, background): “Many techniques have been proposed to solve the problem of catastrophic forgetting (CF) in continual learning (CL), which aims to incrementally learn a sequence of tasks (Chen and Liu 2018).”
CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks (2022, background): “Continual, or lifelong, learning is an essential ability to develop autonomous agents that can learn in a cumulative way [Chen and Liu, 2018].”
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions (2022, background): “Both the duplicate-and-fine-tuning and the feature extraction paradigm have been extensively investigated in the literature, detailed discussions can be found at [6] and we provide a brief comparison.”
TRGP: Trust Region Gradient Projection for Continual Learning (2022, background): “Aiming to achieve this remarkable capability for the deep neural networks (DNNs), continual learning (CL) (Chen & Liu, 2018) has garnered much attention in recent years.”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “Most existing CL models focus on (1), i.e., dealing with catastrophic forgetting or simply forgetting (Chen and Liu, 2018; Parisi et al., 2019; Li and Hoiem, 2016; Seff et al., 2017; Shin et al., 2017; Kirkpatrick et al., 2016; Rebufﬁ et al., 2017; Yoon et al., 2018; He and Jaeger, 2018; Yoon et…”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, background): “In contrast, the lifelong learning (LL) paradigm (Thrun, 1996; Chen & Liu, 2018; Parisi et al., 2019) deﬁnes a biologically inspired learning approach where models learn tasks in sequence, ideally preserving past knowledge and leveraging it to efﬁ-ciently learn new tasks.”
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, background): “Most existing CL papers focus on dealing with CF [21, 5].”
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, background): “Continual learning (CL) has been studied extensively (Chen and Liu, 2018; Parisi et al., 2019).”
Continual Learning for Named Entity Recognition (2021, methodology): “This technique has been used to compress huge language models, such as BERT (Devlin et al. 2019), into smaller ones, e.g., DistilBERT (Sanh et al. 2019).”
Continual Learning by Using Information of Each Class Holistically (2021, background): “Below, we review the other existing approaches (also see the surveys (Chen and Liu 2018; Parisi et al. 2019)).”
Lifelong and Continual Learning Dialogue Systems: Learning during Conversation (2021, methodology): “The first part of the definition is taken from (Chen and Liu 2018) based on the early work in (Thrun 1998; Silver, Yang, and Li 2013; Ruvolo and Eaton 2013; Chen and Liu 2014) and the second part of on-the-job learning is proposed in (Liu 2020).”