Lifelong Domain Word Embedding via Meta-Learning
By Hu Xu, B. Liu, Lei Shu, Philip S. Yu. 2018
39 citations recorded by the source corpus.
Learning high-quality domain word embeddings is important for achieving good performance in many NLP tasks. General-purpose embeddings trained on large-scale corpora are often sub-optimal for domain-specific applications. However, domain-specific tasks often do not have large in-domain corpora for training high-quality domain embeddings. In this paper, we propose a novel lifelong learning setting for domain embedding. That is, when performing the new domain embedding, the system has seen many past domains, and it tries to expand the new in-domain corpus by exploiting the corpora from the past domains via meta-learning. The proposed meta-learner characterizes the similarities of the contexts of the same word in many domain corpora, which helps retrieve relevant data from the past domains to expand the new domain corpus. Experimental results show that domain embeddings produced from such a process improve the performance of the downstream tasks.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- LAMOL: LAnguage MOdeling for Lifelong Language Learning (2019, background): “, 2017), conversational agents (Lee, 2017), word representation learning (Xu et al., 2018), sentence representation learning (Liu et al.”
- Lifelong Language Knowledge Distillation (2020, background): “…for lifelong learning in the previous work, including sentiment analysis (Chen et al., 2015; Xia et al., 2017), conversational agents (Lee, 2017), word and sentence representation learning (Xu et al., 2018; Liu et al., 2019), text classification, and question answering (d’Autume et al., 2019).”
- Open-world Learning and Application to Product Classification (2018, background): “learn initial and easy-to-tune weights for few-shot learning [19, 20], to learn a teacher model that can guide training sample selection [21] and to learn a domain training corpus for word embeddings [22]. Our proposed framework focuses on learning the similarity between an example and an arbitrary ”
- Forward and Backward Knowledge Transfer for Sentiment Classification (2019, background): “Most related work to ours is lifelong learning (LL) (Mitchell et al., 2015; Ruvolo and Eaton, 2013; Chen et al., 2015; Fei et al., 2016; Isele et al., 2016; Xia et al., 2017; Shu et al., 2017; Xu et al., 2018; Sun et al., 2018).”
- Deep Bayesian Unsupervised Lifelong Learning (2021, methodology): “This is in contrast to existing work which uses previous model parameters [5, 20], representative items exacted from previous models [21] or past raw data, or coreset data summarization [16, 17, 18] as previous knowledge for new tasks.”
- Induction Networks for Few-Shot Text Classification (2019, background): “Xu et al. (2018) studied lifelong domain word embeddings via meta-learning.”
- Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction (2018, background): “The proposed embedding mechanism is related to cross domain embeddings (Bollegala et al., 2015, 2017) and domain-specific embeddings (Xu et al., 2018a,b).”
- RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, background): “Learning text classification with few labeled examples has been a problem of interest in many applications (Xu et al., 2018; Yu et al., 2018).”
- LAMAL: LAnguage Modeling Is All You Need for Lifelong Language Learning (2019, methodology): “…studied including sentiment analysis (Chen, Ma, and Liu 2015; Xia, Jiang, and He 2017), conversational agent (Lee 2017), word representation learning (Xu et al. 2018), sentence representation learning (Liu, Ungar, and Sedoc 2019), text classification, and question answering (d’Autume et al. 2019).”