Lifelong Learning for Sentiment Classification
By Zhiyuan Chen, Nianzu Ma, B. Liu. 2015
159 citations recorded by the source corpus.
This paper proposes a novel lifelong learning (LL) approach to sentiment classification. LL mimics the human continuous learning process, i.e., retaining the knowledge learned from past tasks and use it to help future learning. In this paper, we first discuss LL in general and then LL for sentiment classification in particular. The proposed LL approach adopts a Bayesian optimization framework based on stochastic gradient descent. Our experimental results show that the proposed method outperforms baseline methods significantly, which demonstrates that lifelong learning is a promising research direction.
Subfields: Artificial Intelligence, Sentiment & Opinion Analysis, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Sentiment & Opinion Analysis
In Meta & Continual Learning
Citations in the corpus
- ConTinTin: Continual Learning from Task Instructions (2022, background): “Typical work in NLP includes sentiment classification (Chen et al., 2015; Xia et al., 2017), conversational agents (Lee, 2017), text classification, and question answering (d’Autume et al., 2019), etc. TCL tries to learn distinct tasks sequentially.”
- Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, methodology): “However, these lifelong learning works mainly use the traditional learning methods such as regression (Ruvolo and Eaton, 2013), naive Bayes (Chen et al., 2015; Wang et al., 2019), and KNN (Thrun, 1998) to build a model for each task independently and hence there is no forgetting problem.”
- Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “[6] and [61] proposed two Naive Bayesian methods to help improve the new task learning.”
- Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021, background): “Chen et al. (2015) and Wang et al.”
- Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, background): “The current state-of-the-art deep learning approaches perform well on many individual databases [18,55], but suffer from catastrophic forgetting when attempting to learn data associated with new tasks [3,13,19,51,62,65].”
- LAMOL: LAnguage MOdeling for Lifelong Language Learning (2019, background): “The model alleviates the forgetting problem by GEM method, and if it fails to learn the current task and has not been expanded yet, the model will expand to a larger model by the Net2Net approach.”
- Psycholinguistics Meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering (2019, background): “Early work on life-long learning (Chen et al., 2015; Mitchell et al., 2015) is related to ours, but typically concerns a single task (e.g., relation extraction).”
- Sentence Embedding Alignment for Lifelong Relation Extraction (2019, methodology): “long learning is also widely used in (Chen et al., 2015; Chen, 2015; Shu et al., 2016, 2017), which mainly focus on how to represent, reserve and extract knowledge of previous tasks.”
- Lifelong Language Knowledge Distillation (2020, background): “Hence, various NLP tasks have been studied for lifelong learning in the previous work, including sentiment analysis (Chen et al., 2015; Xia et al., 2017), conversational agents (Lee, 2017), word and sentence representation learning (Xu et al., 2018; Liu et al., 2019), text classification, and…”
- Continual Learning with Knowledge Transfer for Sentiment Classification (2021, background): “Lifelong Learning for Sentiment Classification (LSC), proposed by [5], is a Naive Bayes based lifelong learning approach to SC.”
- Learning on the Job: Online Lifelong and Continual Learning (2020, methodology): “Some heuristics have been tried (Chen, Ma and Liu, 2015).”
- Learning Cumulatively to Become More Knowledgeable (2016, background): “Lifelong learning has also been conducted in reinforcement learning [5], and unsupervised topic modeling [8, 9], which use the knowledge extracted from past documents of many domains to improve topic discovery in future tasks.”
- Lifelong Learning CRF for Supervised Aspect Extraction (2017, methodology): “Supervised LML methods exist (Chen et al., 2015; Ruvolo and Eaton, 2013), but they are for classification rather than for sequence learning or labeling like CRF.”
- CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021, methodology): “Two Naive Bayes (NB) approaches were proposed to improve the new task learning (Chen et al., 2015; Wang et al., 2019).”
- Forward and Backward Knowledge Transfer for Sentiment Classification (2019, methodology): “, 2015) used all reviews for training because we need to test on past domains, while (Chen et al., 2015) does not do that.”
- Towards Knowledgeable Supervised Lifelong Learning Systems (2020, unknown): “, 2013) (Ruvolo & Eaton, 2013b) (Pentina & Lampert, 2014) (Chen & Liu, 2016) (Chen et al., 2015) (Parisotto, Ba, & Salakhutdinov, 2015) (Fei et al.”
- Lifelong machine learning: a paradigm for continuous learning (2017, methodology): “[1] proposed an LML technique in the context of naive Bayesian classification.”