Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification

By Tianyu Gao, Xu Han, Zhiyuan Liu, Maosong Sun. 2019

398 citations recorded by the source corpus.

The existing methods for relation classification (RC) primarily rely on distant supervision (DS) because large-scale supervised training datasets are not readily available. Although DS automatically annotates adequate amounts of data for model training, the coverage of this data is still quite limited, and meanwhile many long-tail relations still suffer from data sparsity. Intuitively, people can grasp new knowledge by learning few instances. We thus provide a different view on RC by formalizing RC as a few-shot learning (FSL) problem. However, the current FSL models mainly focus on low-noise vision tasks, which makes them hard to directly deal with the diversity and noise of text. In this paper, we propose hybrid attention-based prototypical networks for the problem of noisy few-shot RC. We design instancelevel and feature-level attention schemes based on prototypical networks to highlight the crucial instances and features respectively, which significantly enhances the performance and robustness of RC models in a noisy FSL scenario. Besides, our attention schemes accelerate the convergence speed of RC models. Experimental results demonstrate that our hybrid attention-based models require fewer training iterations and outperform the state-of-the-art baseline models. The code and datasets are released on https://github.com/thunlp/ HATT-Proto.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification (2020, methodology): “We conduct experiments1 on a large-scale public dataset (i.e., FewRel) to evaluate the effectiveness of our proposed model.”
FLEX: Unifying Evaluation for Few-Shot NLP (2021, background): “Class transfer has been studied on many text classification tasks [5], test including relation classification [25, 28, 64], intent classification [37, 64], inter alia.”
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification (2019, methodology): “Many efforts (Gao et al., 2019; Soares et al., 2019) have been devoted to the new task and some of the methods even exceed human performance 1 on FewRel.”
A Survey on Text Classification: From Traditional to Deep Learning (2020, background): “2019 HAPN [124] EMNLP RC Accuracy FewRel, CSID 2019 Proto-HATT [125] AAAI RC [126] Accuracy FewRel 2019 STCKA [127] AAAI SA, TL [128] Accuracy Weibo, Product Review, etc.”
Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification (2019, background): “Gao et al. (2019) proposed hybrid attention-based pro-totypical networks to handle noisy training samples in few-shot learning.”
Meta-learning for Few-shot Natural Language Processing: A Survey (2020, methodology): “(2019) and Gao et al. (2019a) studied few-shot relation classification problems.”
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021, background): “In addition, in order to accurately represent relations with limited samples, the idea of prototypical networks is introduced into RE(Gao et al., 2019; Ding et al., 2021).”
Finding Task-Relevant Features for Few-Shot Learning by Category Traversal (2019, positive): “[9] applied masks to features in a prototypical network applied to a NLP few-shot sentence classification task, but base their masks only on examples within each class, not between classes as our method does.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, background): “HATT-Proto [184] AAAI’19 Episodic Training Euclidean Distance Designs hybrid networks of attention for instance and prototype features”
Attribute Prototype Network for Zero-Shot Learning (2020, background): “Prototype learning is considered to be more robust when handling open-set recognition [46, 32], out-of-distribution samples [3] and few-shot learning [33, 11, 28].”