Compositional Language Continual Learning

By Yuanpeng Li, Liang Zhao, Kenneth Ward Church, Mohamed Elhoseiny. 2019

40 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Continual Training of Language Models for Few-Shot Learning (2022, methodology): “It has been used for slot ﬁlling (Shen et al., 2019), language learning (Li et al., 2019), sentence embedding (Liu et al., 2019a), translation (Khayral-lah et al., 2018), cross-lingual modeling (Liu et al., 2020b), question answering (Greco et al., 2019) and text classiﬁcation (Ke et al., 2021a,b;…”
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, background): “For example, CL models have been proposed for sentiment analysis [23, 24, 37, 43], dialogue slot filling [53], language modeling [58, 7], language learning [31], sentence embedding [33], machine translation [25], cross-lingual modeling [35], and question answering [12].”
Representational Continuity for Unsupervised Continual Learning (2021, background): “, 2019) and conversational agents (Li et al., 2020).”
Online Coreset Selection for Rehearsal-based Continual Learning (2021, neutral): “In light of this need, continual learning (CL) [40], or lifelong learning, tackles a learning scenario where a model continuously learns over a sequence of tasks [18, 25] within a broad research area, such as classification [15, 9], image generation [47], language learning [24, 4], clinical applicat”
Rainbow Memory: Continual Learning with a Memory of Diverse Samples (2021, background): “This makes continual learning particularly challenging; most AI models suffer from under real-world application scenarios across various domains [38, 20, 30].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “Moreover, multiple works are proposed to address general CL language learning [138, 139] and specific language tasks, such as dialogue systems [140, 141, 142], image captioning [143], sentiment classification [144] and sentence representation learning [145].”
Continual learning in recurrent neural networks (2020, background): “Indeed, some studies report promising results in the context of natural language processing (NLP) (Madasu and Rao, 2020; Thompson et al., 2019), while others ﬁnd that it performs poorly (Asghar et al., 2020; Cossu et al., 2020a; Li et al., 2020).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “One method in this space operates in the lifelong setting, where the vocabulary grows over time (Li et al., 2020b).”
Visually Grounded Continual Learning of Compositional Semantics (2020, background): “Li et al. (2020) is a closely related works that studies challenges in continual learning of sequential prediction tasks while focusing on synthetic instruction following tasks.”
Organizing recurrent network dynamics by task-computation to enable continual learning (2020, background): “Recently, approaches to continual learning based on dynamical network architectures have also been developed in the RNN setting [23, 24].”
Visually Grounded Continual Learning of Compositional Phrases (2020, background): “In particular, Li et al. (2020) study a closely related task of continual learning of sequence prediction for synthetic instruction following.”
Meta-Learning with Sparse Experience Replay for Lifelong Language Learning (2020, background): “In natural language processing (NLP), continual learning still remains relatively unexplored (Li et al., 2020).”
Rethinking the Representational Continuity: Towards Unsupervised Continual Learning (2021, background): “We argue that this assumption is not realistic for most of the real-time applications, including self-driving cars (Bojarski et al., 2016), robotics (Li et al., 2020a), medical applications (Kelly et al., 2019) and conversational agents (Li et al., 2020b).”
Continual Pre-training of Language Models (2023, methodology): “In NLP, CL has been used for slot ﬁlling (Shen et al., 2019), language learning (Li et al., 2019), sentiment analysis (Ke et al., 2021a), topic modeling (Gupta et al., 2020), question answering (Greco et al., 2019) and text classiﬁcation (Sun et al., 2020; Huang et al., 2021; Chuang et al., 2020).”