AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples

By Dongyeop Kang, Tushar Khot, Ashish Sabharwal, Eduard H. Hovy. 2018

86 citations recorded by the source corpus.

We consider the problem of learning textual entailment models with limited supervision (5K-10K training examples), and present two complementary approaches for it. First, we propose knowledge-guided adversarial example generators for incorporating large lexical resources in entailment models via only a handful of rule templates. Second, to make the entailment model—a discriminator—more robust, we propose the first GAN-style approach for training it using a natural language example generator that iteratively adjusts to the discriminator’s weaknesses. We demonstrate effectiveness using two entailment datasets, where the proposed methods increase accuracy by 4.7% on SciTail and by 2.8% on a 1% sub-sample of SNLI. Notably, even a single hand-written rule, negate, improves the accuracy of negation examples in SNLI by 6.1%.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Survey of Data Augmentation Approaches for NLP (2021, methodology): “Kang et al. (2018) and Glockner et al. (2018) create AVEs for textual entailment using WordNet relations.”
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering (2020, background): “One can select the data based on a model’s prediction inconsistencies (Minervini and Riedel, 2018) or randomly sample at each epoch (Kang et al., 2018).”
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, background): “…Data Improvements (Yanaka et al., 2019b) SNLI Stress-test (Glockner et al., 2018; Nie et al., 2019a; Richardson et al., 2019) Data Improvements (Kang et al., 2018; Mitra et al., 2020; Kaushik et al., 2020) Heuristics (Gururangan et al., 2018; Poliak et al., 2018; Zhang et al., 2019a; Nie et…”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, background): “Text Entailment [54, 57, 92, 156] SNLI, SciTail, MultiNLI, SICK”
Dialog State Tracking with Reinforced Data Augmentation (2019, methodology): “While the instance-level reward assigns different reward values to the instances in the same sampled bags by checking whether the instance can cause the Tracker to make incorrect prediction (Kang et al. 2018; Ribeiro, Singh, and Guestrin 2018).”
On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference (2019, methodology): “…a ﬁre truck A man is doing tricks on a skateboard Nobody is doing tricks Although recent work has applied adversarial learning to NLI (Minervini & Riedel, 2018; Kang et al., 2018), this is the ﬁrst work to our knowledge that explicitly studies NLI models designed to ignore hypothesis-only biases.”
Improving Natural Language Inference Using External Knowledge in the Science Questions Domain (2018, background): “Another interesting alternative direction for using KGs such as WordNet for NLI is presented by Kang et al. (2018).”
Look at the First Sentence: Position Bias in Question Answering (2020, background): “For instance, sentence classiﬁcation models in natural language inference and argument reasoning comprehension suffer from word statistics bias (Poliak et al., 2018; Minervini and Riedel, 2018; Kang et al., 2018; Belinkov et al., 2019; Niven and Kao, 2019).”
A Logic-Driven Framework for Consistency of Neural Models (2019, background): “Finally, adversarially labeled examples have been shown to improve prediction accuracy (Kang et al., 2018) .”
Learning from Rules Generalizing Labeled Exemplars (2020, background): “There is growing interest in learning from such efﬁcient, albiet noisy, supervision (Ratner et al., 2016; Pal & Balasubramanian, 2018; Bach et al., 2019; Sun et al., 2018; Kang et al., 2018).”
Data Augmentation Approaches in Natural Language Processing: A Survey (2021, methodology): “Non-Pretrained [26], [27], [80], [122], [123] [13], [79], [124], [125], [78] [80], [126]”
Testing the Generalization Power of Neural Network Models across NLI Benchmarks (2018, background): “Kang et al. (2018) recognize the generalization problem that comes with training on datasets like SNLI, which tend to be homogeneous with linguistic variation.”
Teaching Syntax by Adversarial Distraction (2018, background): “Other work (Kang et al., 2018; Zhao et al., 2018) aimed to improve model robustness in”
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge (2019, background): “Most of this work looks at constructing data through adversarial generation methods, which have also been found useful for creating stronger models (Kang et al., 2018).”