Permutation Equivariant Models for Compositional Generalization in Language

By Jonathan Gordon, David Lopez-Paz, Marco Baroni, Diane Bouchacourt. 2020

112 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Least-to-Most Prompting Enables Complex Reasoning in Large Language Models (2022, background): “, 2019), the alignment of input and output as span trees (Herzig & Berant, 2021), and the permutation equivariance of input and output words (Gordon et al., 2020).”
A Benchmark for Systematic Generalization in Grounded Language Understanding (2020, methodology): “Meta-learning (Lake, 2019) or equivariance permutation (Gordon et al., 2020) could support compositional generalization, if the types of generalizations examined here can be incorporated into a suitable meta-training procedure or an equivariance deﬁnition.”
Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing (2022, background): “…Akyürek et al., 2021; Oren et al., 2021; Qiu et al., 2022; Yang et al., 2022), specialized architectures (Li et al., 2019; Russin et al., 2019; Gordon et al., 2020; Liu et al., 2020; Nye et al., 2020; Chen et al., 2020; Zheng and Lapata, 2021; Oren et al., 2020; Herzig and Berant, 2021; Ruiz…”
Compositional Generalization by Learning Analytical Expressions (2020, methodology): “(ii) Using Extra Resources consists of Good Enough Compositional Data Augmentation (GECA) [2], meta sequence to sequence (Meta Seq2seq) [19], equivariant sequence to sequence (Equivariant Seq2seq) [14] and program synthesis [28].”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Gordon et al. (2020) equated language composition to equivariance on permutations over group actions, and designed an architecture that maintains such equivariances.”
The EOS Decision and Length Extrapolation (2020, methodology): “In NLP, while better modeling techniques have improved models’ abilities to perform some kinds of systematic generalization (Gordon et al., 2019; Lake, 2019), these models still struggle to extrapolate; they have trouble producing and processing sequences longer than those seen during training even ”
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations (2021, unknown): “These include new or modiﬁed model architectures (Li et al., 2019; Russin et al., 2019; Gordon et al., 2020; Liu et al., 2020; Nye et al., 2020; Chen et al., 2020; Zheng and Lapata, 2020; Oren et al., 2020; Herzig and Berant, 2020), hybrid models (Shaw et al., 2020), meta-learning (Lake, 2019), and…”
Span-based Semantic Parsing for Compositional Generalization (2020, background): “…creating new datasets that require compositional generalization (Keysers et al., 2020; Lake and Baroni, 2018; Bah-danau et al. , 2019), and proposing specialized architectures mainly for the SCAN task (Lake, 2019; Nye et al., 2020; Gordon et al., 2020; Liu et al., 2020; Gupta and Lewis, 2018).”
Sequence-to-Sequence Learning with Latent Neural Grammars (2021, unknown): “…has since been a large body of work on compositional sequence-to-sequence learning through various approaches including modiﬁcations to existing architectures [70, 94, 44, 17, 26], grammars and neuro-symbolic models [86, 98, 85, 19, 76], meta-learning [67, 25], and data augmentation [6, 49, 50, 4].”
Teaching Algorithmic Reasoning via In-context Learning (2022, methodology): “…and modifying the architecture align to the algorithms under consideration (Kaiser and Sutskever, 2015; Chiang and Chen, 2018; Xu et al., 2019; Gordon et al., 2019; Yan et al., 2020; Chen et al., 2020; Xhonneux et al., 2021; Veliˇckovi´c and Blundell, 2021) or to the input format (Thawani et…”
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation (2020, methodology): “While several models have made significant improvements over other SCAN splits, progress on the length split remains minimal (Li et al., 2019; Lake, 2019; Gordon et al., 2020).”
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? (2020, methodology): “This has motivated many specialized architectures with stronger compositional biases that improve peformance on SCAN (Li et al., 2019; Russin et al., 2019; Gordon et al., 2019; Lake, 2019; Liu et al., 2020; Nye et al., 2020; Chen et al., 2020).”
Learning to Recombine and Resample Data for Compositional Generalization (2020, methodology): “While details vary, a common feature of existing attempts to formalize systematicity in sequence modeling problems (e.g. Gordon et al., 2020) is the intuition that learners should make accurate predictions in situations featuring novel combinations of previously observed input or output…”
Compositional Generalization via Neural-Symbolic Stack Machines (2020, methodology): “…99.8 CNN [12] 0.0 56.7 69.2 100.0 Syntactic Attention [40] 15.2 28.9 91.0 − Primitive Substitution [28] 20.3 83.2 98.8 99.9 Equivariant seq2seq [16] 15.9 92.0 99.1 100.0 [25] 16.64 98.71 99.95 − Synth (no search) [36] 0.0 0.0 3.5 13.3 Synth (with search) [36] 100.0 100.0 100.0 100.0 Table 4:…”
Compositional Semantic Parsing with Large Language Models (2022, background): “…of neural model architectures (Chen et al., 2020; Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021; Russin et al., 2019; Li et al., 2019; Gordon et al., 2020; Herzig & Berant, 2021) and training algorithms (Lake, 2019; Kim, 2021), training data augmentation (Andreas, 2020; Akyürek et…”