Adversarial Training for Relation Extraction

By Yi Wu, David Bamman, Stuart J. Russell. 2017

226 citations recorded by the source corpus.

Adversarial training is a mean of regularizing classification algorithms by generating adversarial noise to the training data. We apply adversarial training in relation extraction within the multi-instance multi-label learning framework. We evaluate various neural network architectures on two different datasets. Experimental results demonstrate that adversarial training is generally effective for both CNN and RNN models and significantly improves the precision of predicted relations.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “which in turn helps to increase robustness and performance of neural networks [24] and improve the interpretability [25].”
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, methodology): “…proposed by Lin et al. (2016), which is followed by the recent state-of-the-art methods (Zeng et al., 2017; Ji et al., 2017; Huang and Wang, 2017; Wu et al., 2017; Liu et al., 2017; Feng et al., 2018; Zeng et al., 2018), on the benchmark dataset NYT-10 NYT-10 ( Riedel We formulate RC as a…”
Towards Understanding Gender Bias in Relation Extraction (2019, methodology): “predictions of a collection of popularly used and open source NRE models1 (Lin et al., 2016; Wu et al., 2017; Liu et al., 2017; Feng et al., 2018).”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, methodology): “ i.e., the word embedding of the adversariesshould be valid word embeddings in the vocabulary, they introduced a direction vector which associates the perturbed embedding to the valid word embedding. [145] simply adopts the regularizer utilized in [94], but applies the perturbations on pre-trained w”
Domain-agnostic Question-Answering with Adversarial Training (2019, background): “It can be extended to text classiﬁcation (Chen et al., 2016; Liu et al., 2017; Chen and Cardie, 2018), and relation extraction (Wu et al., 2017).”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “Adversarial Training (AT) is applied to add adversarial noise to word embeddings for CNN- and RNN-based relation extraction under the MIML learning setting [125].”
Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks (2019, methodology): “We evaluate our models on the NYT dataset developed by (Riedel et al., 2010), which has been widely used in recent studies (Lin et al., 2016; Liu et al., 2017; Wu et al., 2017; Feng et al., 2018).”
Adversarial Personalized Ranking for Recommendation (2018, methodology): “Thus, we employ the fast gradient method proposed in Goodfellow et al. [15], a common choice in adversarial training [24, 26, 34].”
Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification (2019, background): “Because DS brings inevitable noise in itself, many efforts are devoted to further reducing noise (Wu, Bamman, and Russell 2017; Feng et al. 2018).”
Multi-label Adversarial Perturbations (2018, background): “It also motivates the explorations of generating powerful perturbations to analyze the vulnerability of machine learning models, detect their blind spots, and improve their robustness [8]–[10].”
ERNIE: Enhanced Language Representation with Informative Entities (2019, unknown): “are introduced into CNN (Zeng et al., 2015; Lin et al., 2016; Wu et al., 2017; Han et al., 2018b).”
DocRED: A Large-Scale Document-Level Relation Extraction Dataset (2019, background): “In recent years, various neural models have been explored to encode relational patterns of entities for sentence-level RE, and achieve state-of-theart performance (Socher et al., 2012; Zeng et al., 2014, 2015; dos Santos et al., 2015; Xiao and Liu, 2016; Cai et al., 2016; Lin et al., 2016; Wu et al.”
Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient (2019, background): “2017a; OpenAI 2018), traffic control (Wu et al. 2017) and the analysis of social dilemmas (Leibo et al.”
DARE: Data Augmented Relation Extraction with GPT-2 (2020, background): “In this work, we focus on supervised RE (Zeng et al., 2014; Lin et al., 2016; Wu et al., 2017; Verga et al., 2018), where relation types come from a set of predeﬁned categories, as opposed to Open Information Extraction approaches that represent relations among entities using their surface forms…”
Prototypical Representation Learning for Relation Extraction (2021, unknown): “…-10.900574 32.073906 -7.5629783 36.904022 -15.117426 -47.537777 , various denoising methods have been explored to reduce noise effects for distant supervision (Zeng et al., 2015; Lin et al., 2016; Jiang et al., 2016; Han et al., 2018a; Wu et al., 2017; Qin et al., 2018a;b; Feng et al., 2018).”
Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention (2018, methodology): “…over instances (Lin et al., 2016); +ATT+ADV is the denoising attention method by adding a small adversarial perturbation to instance embeddings (Wu et al., 2017); +ATT+SL is the attention-based model using soft-labeling method to mitigate the side effect of the wrong labeling problem at…”
Robust Multilingual Part-of-Speech Tagging via Adversarial Training (2017, background): “Note that α =0 would generate no noise (identical to the baseline); if α = 1 , the generated adversarial perturbation would have a norm comparable to the original embedding, which could change the semantics of the input sentence (Wu et al., 2017).”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “, 2017) DANN 3 (Wu et al., 2017) DANN 3 (Yasunaga et al.”
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, methodology): “sarial training (Wu et al., 2017; Wang et al., 2018; Han et al., 2018a) have also been adopted in DS.”