LAMAL: LAnguage Modeling Is All You Need for Lifelong Language Learning

By Fan-Keng Sun, Cheng-Hao Ho, Hung-yi Lee. 2019

39 citations recorded by the source corpus.

Subfields: Large Language Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

1. Zixuan Ke
2. Bin Liu
3. Hu Xu
4. Lei Shu
5. Bing Liu
6. Yijia Shao
7. Haowei Lin
8. Tatsuya Konishi

In Meta & Continual Learning

1. Bing Liu
2. Zixuan Ke
3. Bin Liu
4. Gyuhak Kim
5. Hu Xu
6. Lei Shu
7. Tatsuya Konishi
8. Nianzu Ma

Citations in the corpus

• Continual Training of Language Models for Few-Shot Learning (2022, background): “…(Liu et al., 2019a), translation (Khayral-lah et al., 2018), cross-lingual modeling (Liu et al., 2020b), question answering (Greco et al., 2019) and text classification (Ke et al., 2021a,b; Sun et al., 2020; Huang et al., 2021; Chuang et al., 2020; Mehta et al., 2021; Madotto et al., 2020).”
• Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, methodology): “LAMOL outperforms BERT-based MTL in DSC.”
• Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, methodology): “Sun et al. (2020a); Wang et al. (2021); Madotto et al. (2021) formulate multiclass-text classification as a task-incremental learning ( Task-IL ) problem where the objective is to train an “incremental learner” on a sequence of tasks.”
• Continual Pre-training of Language Models (2023, methodology): “In NLP, CL has been used for slot filling (Shen et al., 2019), language learning (Li et al., 2019), sentiment analysis (Ke et al., 2021a), topic modeling (Gupta et al., 2020), question answering (Greco et al., 2019) and text classification (Sun et al., 2020; Huang et al., 2021; Chuang et al., 2020).”
• One Person, One Model, One World: Learning Continual User Representation without Forgetting (2020, background): “The major drawback for EWC and LwF is that neither of them has thoroughly settled the forgetting issue (on the CV or reinforcement learning tasks) since when more new tasks are learned, their performance on older task still decrease significantly [15, 32, 38].”
• DARE: Data Augmented Relation Extraction with GPT-2 (2020, methodology): “Chen et al. (2011) A similar method to our approach was proposed by Sun et al. (2019a) who presented a framework to successfully deal with catastrophic forgetting in language lifelong learning (LLL).”
• Neural Data-to-Text Generation with LM-based Text Augmentation (2021, background): “People have also also shown that GPT-2 is able to improve classification tasks via in-domain text augmentation (Papanikolaou and Pierleoni, 2020; Sun et al., 2020).”