Continual Learning with Knowledge Transfer for Sentiment Classification

By Zixuan Ke, Bing Liu, Hao Wang, Lei Shu. 2021

36 citations recorded by the source corpus.

This paper studies continual learning (CL) for sentiment classification (SC). In this setting, the CL system learns a sequence of SC tasks incrementally in a neural network, where each task builds a classifier to classify the sentiment of reviews of a particular product category or domain. Two natural questions are: Can the system transfer the knowledge learned in the past from the previous tasks to the new task to help it learn a better model for the new task? And, can old models for previous tasks be improved in the process as well? This paper proposes a novel technique called KAN to achieve these objectives. KAN can markedly improve the SC accuracy of both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of KAN is demonstrated through extensive experiments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Sentiment & Opinion Analysis, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Zixuan Ke
3. Hu Xu
4. Lei Shu
5. Nianzu Ma
6. Qi Qin
7. B. Liu
8. Martin Wistuba

In Computer Vision and Pattern Recognition

1. Zixuan Ke
2. Bing Liu
3. Hu Xu
4. B. Ermiş
5. Giovanni Zappella
6. Martin Wistuba
7. Aditya Rawal
8. Qi Qin

In Sentiment & Opinion Analysis

1. Zixuan Ke
2. Bing Liu
3. Hu Xu
4. Lei Shu
5. Qi Qin
6. Nianzu Ma
7. Xingchang Huang
8. Wenpeng Hu

In Meta & Continual Learning

1. Bing Liu
2. Zixuan Ke
3. Hu Xu
4. Lei Shu
5. Qi Qin
6. Nianzu Ma
7. Wenpeng Hu
8. Dongyan Zhao

Citations in the corpus

• Memory Efficient Continual Learning with Transformers (2022, background): “Recent work studied catastrophic forgetting [55, 10, 27, 35] and incremental learning [66] for NLP and CV [32, 15].”
• Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, methodology): “Ke et al. (2020) and Wang et al. (2019) also did backward transfer.”
• Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “Although several papers have tried to deal with both [22, 37], they were only tested using sentiment analysis tasks with strong shared knowledge.”
• Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, methodology): “KAN (Ke et al., 2020b) and SRK (Lv et al., 2019) are TIL methods for document sentiment classification.”
• Continual Learning by Using Information of Each Class Holistically (2021, background): “Some applications in sentiment analysis have been reported in (Ke et al. 2020; Qin, Hu, and Liu 2020), which focus on knowledge transfer.”
• Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “Moreover, multiple works are proposed to address general CL language learning [137, 138] and specific language tasks, such as dialogue systems [139, 140, 141], image captioning [142], sentiment classification [143] and sentence representation learning [144].”
• CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021, methodology): “Existing CL systems SRK (Lv et al., 2019) and KAN (Ke et al., 2020b) are for DSC in the TIL setting, not for ASC.”
• Memory Efficient Continual Learning for Neural Text Classification (2022, methodology): “…some studies use pre-trained models for class incremental learning (Ke et al., 2021; Hu et al., 2021); (4) Parameter isolation-based methods (Ke et al., 2020; Worts-man et al., 2020) allocate model parameters dedicated to different tasks and mask them out when learning a new task; (5)…”