Active Lifelong Learning With "Watchdog"

By Gan Sun, Yang Cong, Xiaowei Xu. 2018

35 citations recorded by the source corpus.

Lifelong learning intends to learn new consecutive tasks depending on previously accumulated experiences, i.e., knowledge library. However, the knowledge among different new coming tasks are imbalance. Therefore, in this paper, we try to mimic an effective "human cognition" strategy by actively sorting the importance of new tasks in the process of unknown-to-known and selecting to learn the important tasks with more information preferentially. To achieve this, we consider to assess the importance of the new coming task, i.e., unknown or not, as an outlier detection issue, and design a hierarchical dictionary learning model consisting of two-level task descriptors to sparse reconstruct each task with the l0 norm constraint. The new coming tasks are sorted depending on the sparse reconstruction score in descending order, and the task with high reconstruction score will be permitted to pass, where this mechanism is called as "watchdog." Next, the knowledge library of the lifelong learning framework encode the selected task by transferring previous knowledge, and then can also update itself with knowledge from both previously learned task and current task automatically. For model optimization, the alternating direction method is employed to solve our model and converges to a fixed point. Extensive experiments on both benchmark datasets and our own dataset demonstrate the effectiveness of our proposed model especially in task selection and dictionary learning.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Shuai Wang
3. Zicheng Liu
4. Zixuan Ke
5. Lijuan Wang
6. Nianzu Ma
7. Hao Wang
8. Bernt Schiele

In Computational Mechanics

1. Lijuan Wang
2. Zicheng Liu
3. Kaihua Tang
4. Nianzu Ma
5. Chunyan Miao
6. Yun Raymond Fu
7. Bing Liu
8. Hao Wang

In Computer Vision and Pattern Recognition

1. Bernt Schiele
2. Bing Liu
3. Yun Raymond Fu
4. Zicheng Liu
5. Hanwang Zhang
6. Luc Van Gool
7. Lijuan Wang
8. Trevor Darrell

Citations in the corpus

• Learning with Recoverable Forgetting (2022, neutral): “Apart from those works that still need part of the old data, many researchers are devoted to developing the methods without storing the old data by synthesizing old data [9,41,46] or even without referring to any old data [44,42,32].”
• Large Scale Incremental Learning (2019, background): “[25, 26] updated knowledge dictionary for new tasks and kept dictionary coefficients for old tasks.”
• Forward and Backward Knowledge Transfer for Sentiment Classification (2019, background): “Most related work to ours is lifelong learning (LL) (Mitchell et al., 2015; Ruvolo and Eaton, 2013; Chen et al., 2015; Fei et al., 2016; Isele et al., 2016; Xia et al., 2017; Shu et al., 2017; Xu et al., 2018; Sun et al., 2018).”
• Learning to Segment the Tail (2020, methodology): “To cope with the everevolving data, class-incremental learning methods [36, 16, 38, 2] adapt the original model trained on old classes to new classes, where knowledge distillation [15, 23] and old data replay [33, 26] are applied to minimize the forgetting.”