Using the Past Knowledge to Improve Sentiment Classification

By Qi Qin, Wenpeng Hu, Bing Liu. 2020

11 citations recorded by the source corpus.

This paper studies sentiment classification in the lifelong learning setting that incrementally learns a sequence of sentiment classification tasks. It proposes a new lifelong learning model (called L2PG) that can retain and selectively transfer the knowledge learned in the past to help learn the new task. A key innovation of this proposed model is a novel parameter-gate (p-gate) mechanism that regulates the flow or transfer of the previously learned knowledge to the new task. Specifically, it can selectively use the network parameters (which represent the retained knowledge gained from the previous tasks) to assist the learning of the new task t. Knowledge distillation is also employed in the process to preserve the past knowledge by approximating the network output at the state when task t-1 was learned. Experimental results show that L2PG outperforms strong baselines, including even multiple task learning.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Zixuan Ke
3. Hu Xu
4. Lei Shu
5. Dongyan Zhao
6. Nianzu Ma
7. Qi Qin
8. Wenpeng Hu

Citations in the corpus

• Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “For example, CL models have been proposed for sentiment analysis [23, 24, 37, 43], dialogue slot filling [53], language modeling [58, 7], language learning [31], sentence embedding [33], machine translation [25], cross-lingual modeling [35], and question answering [12].”
• Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, methodology): “Wang et al. (2018) used LL for ASC, but improved only the new task and did not deal with CF. Existing CL systems SRK (Lv et al., 2019), KAN (Ke et al., 2020b) and L2PG (Qin et al., 2020) are for document sentiment classification, but not ASC.”
• Continual Learning by Using Information of Each Class Holistically (2021, background): “Some applications in sentiment analysis have been reported in (Ke et al. 2020; Qin, Hu, and Liu 2020), which focus on knowledge transfer.”
• CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021, methodology): “, the data owners do not want their data used by others (Ke et al., 2020b; Qin et al., 2020; Ke et al., 2021).”
• BNS: Building Network Structures Dynamically for Continual Learning (2021, background): “Several continual sentiment classification methods have focused on knowledge transfer as sentiment tasks are similar [42, 22].”