Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network

By Yutai Hou, Wanxiang Che, Yong Lai, Zhihan Zhou, Yijia Liu, Han Liu, Ting-Wei Liu. 2020

207 citations recorded by the source corpus.

In this paper, we explore the slot tagging with only a few labeled support sentences (a.k.a. few-shot). Few-shot slot tagging faces a unique challenge compared to the other fewshot classification problems as it calls for modeling the dependencies between labels. But it is hard to apply previously learned label dependencies to an unseen domain, due to the discrepancy of label sets. To tackle this, we introduce a collapsed dependency transfer mechanism into the conditional random field (CRF) to transfer abstract label dependency patterns as transition scores. In the few-shot setting, the emission score of CRF can be calculated as a word’s similarity to the representation of each label. To calculate such similarity, we propose a Label-enhanced Task-Adaptive Projection Network (L-TapNet) based on the state-of-the-art few-shot classification model – TapNet, by leveraging label name semantics in representing labels. Experimental results show that our model significantly outperforms the strongest few-shot learning baseline by 14.64 F1 scores in the one-shot setting.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Few-Shot Class-Incremental Learning for Named Entity Recognition (2022, background): “These generally involves predicting by learning to compare token features with class pro-totypes (Hou et al., 2020) or stored query samples (training data) of target classes (Yang and Kati-yar, 2020).”
Neural Data Augmentation via Example Extrapolation (2021, methodology): “Hou et al. (2020) and Krone et al. (2020) both involve explicitly aligning token- or span-vectors from an incoming query to prototype vectors derived from Ftrain and computing the similarity between them directly.”
Document-Level Event Argument Extraction by Conditional Generation (2021, methodology): “Compared with (Hou et al., 2020), we do not collapse the entries of the transition matrix, making it possible for our model to learn different probabilities for each event type.”
Controllable Semantic Parsing via Retrieval Augmentation (2021, methodology): “We particularly focus on three scenarios: (1) Domain bootstrapping: making the parser handle queries in a new domain (Su and Yan, 2017; Hou et al., 2020a; Li et al., 2021b).”
Few-Shot Named Entity Recognition: A Comprehensive Study (2020, methodology): “( iii ) SimBERT is a simple baseline reported in (Yang and Katiyar, 2020; Hou et al., 2020); it utilizes a nearest neighbor classiﬁer based on the contextualized representation output by the pre-trained BERT, without ﬁne-tuning on few-shot examples.”
Few-NERD: A Few-shot Named Entity Recognition Dataset (2021, methodology): “…type of Y and a query x , With the identical basic structure as NNShot, StructShot adopts an additional Viterbi decoder during the inference phase (Hou et al., 2020) p ( y | x ) and solve the problem: To sum up, BERT-Tagger is a well-acknowledged baseline that could produce pronounced results on…”