CNNs found to jump around more skillfully than RNNs: Compositional Generalization in Seq2seq Convolutional Networks

By Roberto Dessì, Marco Baroni. 2019

47 citations recorded by the source corpus.

Lake and Baroni (2018) introduced the SCAN dataset probing the ability of seq2seq models to capture compositional generalizations, such as inferring the meaning of “jump around” 0-shot from the component words. Recurrent networks (RNNs) were found to completely fail the most challenging generalization cases. We test here a convolutional network (CNN) on these tasks, reporting hugely improved performance with respect to RNNs. Despite the big improvement, the CNN has however not induced systematic rules, suggesting that the difference between compositional and non-compositional behaviour is not clear-cut.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Model Architectures

Citations in the corpus

Compositional Generalization via Neural-Symbolic Stack Machines (2020, background): “Length Around right Jump Simple NeSS (ours) 100.0 100.0 100.0 100.0 Seq2seq [24] 13.8 − 0.08 99.8 CNN [12] 0.0 56.7 69.2 100.0 Syntactic Attention [40] 15.2 28.9 91.0 − Primitive Substitution [28] 20.3 83.2 98.8 99.9 Equivariant seq2seq [16] 15.9 92.0 99.1 100.0 [25] 16.64 98.71 99.95 − Synth (no…”
Compositional generalization in a deep seq2seq model by separating syntax and semantics (2019, methodology): “Overall, this represents an improvement over the best previously reported results on this task [4; 8], and does so without any hand-engineered features or additional supervision.”
Compositional Generalization by Learning Analytical Expressions (2020, background): “Subsequent works veriﬁed that it was not an isolated case, since convolutional encoder-decoder model and Transformer met the same problem [10].”
Permutation Equivariant Models for Compositional Generalization in Language (2020, unknown): “In particular, Lake (2019) designs one specific and complex meta-learning procedure for each SCAN task, where a distribution over tasks is provided to the learner (Finn et al., 2017; Gordon et al., 2018). For example, in the Add jump and Around right tasks, the meta-learning procedure of Lake (2019)”
Linguistic generalization and compositionality in modern artificial neural networks (2019, background): “In very recent work, Dess̀ı and Baroni [60] found a somewhat more intriguing pattern.”