Transfer Learning for Sequence Labeling Using Source Model and Target Data

By Lingzhen Chen, Alessandro Moschitti. 2019

34 citations recorded by the source corpus.

In this paper, we propose an approach for transferring the knowledge of a neural model for sequence labeling, learned from the source domain, to a new model trained on a target domain, where new label categories appear. Our transfer learning (TL) techniques enable to adapt the source model using the target data and new categories, without accessing to the source data. Our solution consists in adding new neurons in the output layer of the target model and transferring parameters from the source model, which are then fine-tuned with the target data. Additionally, we propose a neural adapter to learn the difference between the source and the target label distribution, which provides additional important information to the target model. Our experiments on Named Entity Recognition show that (i) the learned knowledge in the source model can be effectively transferred when the target data contains new categories and (ii) our neural adapter further improves such transfer.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Continual Learning for Named Entity Recognition (2021, background): “In terms of CL for NER speciﬁcally, the work which is most related is by Chen and Moschitti (2019), who pro-pose an approach for transferring the knowledge of a neural network for sequence labeling trained on one (source) dataset to a new model trained on another (target) dataset in which new label…”
AdaRNN: Adaptive Learning and Forecasting of Time Series (2021, methodology): “Several methods applied pre-training to RNNs [4, 8, 11, 15, 51] which is similar to CNN-based pre-training.”