Compositional Generalization by Learning Analytical Expressions
By Qian Liu, Shengnan An, Jian-Guang Lou, Zeqi Lin, Yan Gao, Bin Zhou, Nanning Zheng, Dongmei Zhang. 2020
78 citations recorded by the source corpus.
Compositional generalization is a basic but essential intellective capability of human beings, which allows us to recombine known parts readily. However, existing neural network based models have been proven to be extremely deficient in such a capability. Inspired by work in cognition which argues compositionality can be captured by variable slots with symbolic functions, we present a refreshing view that connects a memory-augmented neural model with analytical expressions, to achieve compositional generalization. Our model consists of two cooperative neural modules Composer and Solver, fitting well with the cognitive argument while still being trained in an end-to-end manner via a hierarchical reinforcement learning algorithm. Experiments on a well-known benchmark SCAN demonstrate that our model seizes a great ability of compositional generalization, solving all challenges addressed by previous works with 100% accuracies.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Mohit Bansal
- Percy Liang
- Luke S. Zettlemoyer
- Jonathan Berant
- Quoc V. Le
- Graham Neubig
- Caiming Xiong
- Richard Socher
Citations in the corpus
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models (2022, methodology): “In the prompt constructed to solve SCAN, we actually used both natural language and Python notation.”
- Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? (2020, methodology): “Additionally, we include results for GECA 9 (Andreas, 2020), a data augmentation method, as well as LANE (Liu et al., 2020) and NSSM (Chen et al., 2020) 10 .”
- How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “Another recent technique incorporates a memory of automatically extracted analytical expressions and uses those to compositionally generalize (Liu et al., 2020).”
- Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing (2022, background): “…2021; Oren et al., 2021; Qiu et al., 2022; Yang et al., 2022), specialized architectures (Li et al., 2019; Russin et al., 2019; Gordon et al., 2020; Liu et al., 2020; Nye et al., 2020; Chen et al., 2020; Zheng and Lapata, 2021; Oren et al., 2020; Herzig and Berant, 2021; Ruiz et al., 2021; Wang et…”
- Compositional Semantic Parsing with Large Language Models (2022, background): “However, SCAN is an artificial task built upon a synthetic language with a tiny vocabulary and is generated from a small set of grammar rules, and it is unclear whether strong results transfer to more realistic tasks that are based on a larger vocabulary and more complicated grammars (Furrer et al.,”