The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models

By Noah Weber, Leena Shekhar, Niranjan Balasubramanian. 2018

30 citations recorded by the source corpus.

Seq2Seq based neural architectures have become the go-to architecture to apply to sequence to sequence language tasks. Despite their excellent performance on these tasks, recent work has noted that these models typically do not fully capture the linguistic structure required to generalize beyond the dense sections of the data distribution (Ettinger et al., 2017), and as such, are likely to fail on examples from the tail end of the distribution (such as inputs that are noisy (Belinkov and Bisk, 2018), or of different length (Bentivogli et al., 2016)). In this paper we look at a model’s ability to generalize on a simple symbol rewriting task with a clearly defined structure. We find that the model’s ability to generalize this structure beyond the training distribution depends greatly on the chosen random seed, even when performance on the test set remains the same. This finding suggests that model’s ability to capture generalizable structure is highly sensitive, and more so, this sensitivity may not be apparent when evaluating the model on standard test sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (2019, background): “This effect can arise not just from different architectures but also from different initializations of the same architecture (Weber et al., 2018).”
BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance (2019, methodology): “Weber et al. (2018) trained 50 instances of a sequence-to-sequence model on a simple symbol replacement task and found that all 50 instances achieved over 99% accuracy on the in-distribution test set but had highly variable performance on out-of-distribution generalization sets; for example, in the…”
Learning compositionally through attentive guidance (2018, methodology): “C L ] 7 S ep 2 01 8 Kruszewski, and Baroni 2018) and the symbol rewriting task (Weber, Shekhar, and Balasubramanian 2018).”
Location Attention for Extrapolation to Longer Sequences (2019, background): “This issue has recently been uncovered for seq2seq models by looking at simple artificial tasks (Lake and Baroni, 2018; Liska et al., 2018; Weber et al., 2018).”
White-to-Black: Efficient Distillation of Black-Box Adversarial Attacks (2019, methodology): “Adversarial examples have been extensively used recently in NLP for probing and understanding neural models (Jia and Liang, 2017; Weber et al., 2018).”
RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency (2018, background): “…al., 2016; Gulordava et al., 2018; Wilcox et al., 2018) and complements a variety of other approaches currently practiced (Shi et al., 2016; Belinkov et al., 2018; Blevins et al., 2018; K´ad´ar et al., 2017; Williams et al., 2018; Et-tinger et al., 2017; Lake and Baroni, 2017; Weber et al., 2018).”
The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions (2020, background): “Weber et al. (2018) found that the model’s ability to generalize beyond the training distribution depends greatly on the random seed.”