Teaching Syntax by Adversarial Distraction

By Juho Kim, Christopher Malon, Asim Kadav. 2018

6 citations recorded by the source corpus.

Existing entailment datasets mainly pose problems which can be answered without attention to grammar or word order. Learning syntax requires comparing examples where different grammar and word order change the desired classification. We introduce several datasets based on synthetic transformations of natural entailment examples in SNLI or FEVER, to teach aspects of grammar and word order. We show that without retraining, popular entailment models are unaware that these syntactic differences change meaning. With retraining, some but not all popular entailment models can learn to compare the syntax properly.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (2019, methodology): “s typical classiﬁcation approaches only work for models with vector representations. 8.2 Structural heuristics Similar to our lexical overlap heuristic, Dasgupta et al. (2018), Nie et al. (2018), and Kim et al. (2018) also tested NLI models on speciﬁc phenomena where word order matters; we use a lar”
Syntactic Data Augmentation Increases Robustness to Inference Heuristics (2020, methodology): “Finally, Kim et al. (2018) used a similar augmentation approach to ours but did not study generalization to types of examples not in the augmentation set.”
What Happens To BERT Embeddings During Fine-tuning? (2020, background): “behavioral studies with challenge sets (McCoy et al., 2019b; Poliak et al., 2018; Ettinger et al., 2018; Kim et al., 2018) have shown limited ability to generalize to out-of-domain data and across syntactic perturbations.”