Meta-Learning Improves Lifelong Relation Extraction

By Abiola Obamuyide, Andreas Vlachos. 2019

32 citations recorded by the source corpus.

Most existing relation extraction models assume a fixed set of relations and are unable to adapt to exploit newly available supervision data to extract new relations. In order to alleviate such problems, there is the need to develop approaches that make relation extraction models capable of continuous adaptation and learning. We investigate and present results for such an approach, based on a combination of ideas from lifelong learning and optimization-based meta-learning. We evaluate the proposed approach on two recent lifelong relation extraction benchmarks, and demonstrate that it markedly outperforms current state-of-the-art approaches.

Subfields: Artificial Intelligence, Information Extraction, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Extraction

In Meta & Continual Learning

Citations in the corpus

Memory Efficient Continual Learning with Transformers (2022, methodology): “…21]; (4) Parameter isolation-based methods [25, 62] allocate model parameters dedicated to different tasks and mask them out when learning a new task; (5) Meta-learning-based methods, which directly optimize the knowledge transfer among tasks [47, 37] or learn robust data representations [24, 60].”
Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation (2022, neutral): “, 2020) and continual relation learning (Wang et al., 2019; Obamuyide and Vlachos, 2019; Han et al., 2020; Wu et al., 2021).”
Curriculum-Meta Learning for Order-Robust Continual Relation Extraction (2021, methodology): “(See Appendix C for the result for EMAR, MLLRE, and CML.) As shown in Table ??, comparing the results in the rows of P0 and P9 of each task, we can observe that most tasks see a significant drop in accuracy from P9 (when the task is the last one seen by the model) to P0 (when the task is the first o”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, methodology): “…Classification Dataset • sentence-relation pairs derived from Wikipedia distributed over 10 disjoint clusters (representing different tasks) (Wang et al., 2019b) (Obamuyide and Vlachos, 2019) trained on a large unlabeled text data; (2) this pre-trained language representation model is then reused…”
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, neutral): “…) model where, similar to existing approaches (de Masson d'Autume et al., 2019; Wang et al., 2020; Qian et al., 2021; Xia et al., 2021; Wang et al., 2019; Obamuyide and Vlachos, 2019; Han et al., 2020) we do not store real samples in replay memory but instead store concept words for each intent.”
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021, background): “In recent years, some memory-based CRE models have made signiﬁcant progress in overcoming catastrophic forgetting while learning new relations, such as EA-EMR (Wang et al., 2019), MLLRE (Obamuyide and Vlachos, 2019), CML (Wu et al., 2021) and EMAR (Han et al., 2020).”
Memory Efficient Continual Learning for Neural Text Classification (2022, methodology): “…dedicated to different tasks and mask them out when learning a new task; (5) Meta-learning-based methods, which directly optimize the knowledge transfer among tasks (Riemer et al., 2018; Obamuyide et al., 2019) or learn robust data representations (Javed & White, 2019; Wang et al., 2020).”
Meta-Learning with Sparse Experience Replay for Lifelong Language Learning (2020, methodology): “experience replay (Riemer et al., 2019; Obamuyide and Vlachos, 2019a) (in image classification, reinforcement learning and language processing).”