Compositionality Decomposed: How do Neural Networks Generalise?

By Dieuwke Hupkes, Verna Dankers, Mathijs Mul, Elia Bruni. 2019

408 citations recorded by the source corpus.

Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack of agreement about what it means for a neural model to be compositional. As a response to this controversy, we present a set of tests that provide a bridge between, on the one hand, the vast amount of linguistic and philosophical theory about compositionality of language and, on the other, the successful neural models of language. We collect different interpretations of compositionality and translate them into five theoretically grounded tests for models that are formulated on a task-independent level. In particular, we provide tests to investigate (i) if models systematically recombine known parts and rules (ii) if models can extend their predictions beyond the length they have seen in the training data (iii) if models’ composition operations are local or global (iv) if models’ predictions are robust to synonym substitutions and (v) if models favour rules or exceptions during training. To demonstrate the usefulness of this evaluation paradigm, we instantiate these five tests on a highly compositional data set which we dub PCFG SET and apply the resulting tests to three popular sequence-to-sequence models: a recurrent, a convolution-based and a transformer model. We provide an in-depth analysis of the results, which uncover the strengths and weaknesses of these three architectures and point to potential areas of improvement.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Scaling & Learning Theory

Citations in the corpus

Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021, methodology): “Second, the trained models usually suffer from the compositionality generalization problem, meaning that they tend to fail when the number of reasoning steps are much larger in the evaluation set than in the training set (Hupkes et al., 2020; Hahn, 2020; Clark et al., 2020).”
Scaling Laws for Autoregressive Generative Modeling (2020, background): “Mathematical problem solving lends itself very naturally to the study of extrapolation, because we can extend the range of numbers or operations used to create math problems, or the recursive/compositional depth [HDMB19] required for a solution.”
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “This is driven by the compositional generality of language (Hupkes et al., 2020; Keysers et al., 2019), and can be applied to domains in which data is scarce, models are only available through APIs without source-code access, or it may be prohibitively expensive to train a new large multimodal model”
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2021, methodology): “, 2021), sequence-to-sequence models trained on an artificial dataset (Hupkes et al., 2020), pretrained sequence-to-sequence models tested on arithmetic tasks (Nogueira et al.”
On the Binding Problem in Artificial Neural Networks (2020, background): “For example, it is difficult for neural language models to generalize syntactic rules such as verb tenses or embedded clauses in a systematic manner (Keysers et al., 2020; Lake and Baroni, 2018; Loula et al., 2018; Hupkes et al., 2020).”
Measuring and Narrowing the Compositionality Gap in Language Models (2022, positive): “We are the first to propose the use of 2-hop questions as a means of evaluating the compositional abilities of large LMs. Lake and Baroni (2017) and Hupkes et al. (2020) previously tested the ability of sequence-to-sequence models trained on an artificial dataset to integrate known parts into novel…”
Can Wikipedia Help Offline Reinforcement Learning? (2022, background): “…look to investigate in more depth which properties of language structure are useful for reinforcement learning and sequence modeling in other domains, and whether previous work studying language structure (Hupkes et al., 2019) does indeed relate to compositional generalization of neural networks.”
Understanding Natural Language Understanding Systems. A Critical Analysis (2023, background): “…is supported by experiments that reveal that the most recent neural models trained on simplified or artificial language data show generalization abilities but fail to generalize in a systematic way (Lake and Baroni 2018, Goodwin, Sinha and Donnell 2020, Hupkes, Dankers, Mul and Bruni 2020).”
The Impact of Positional Encoding on Length Generalization in Transformers (2023, background): “(Anil et al., 2022), LEGO (Zhang et al., 2023) and (3) Classical length generalization datasets such as SCAN (Lake and Baroni, 2018) and PCFG (Hupkes et al., 2020).”
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks (2020, background): “It provides a natural way of achieving compositionality, which appears essential for systematic generalization, one of the areas where typical artificial neural networks (NNs) do not yet perform well (Fodor et al., 1988; Marcus, 1998; Lake & Baroni, 2018; Hupkes et al., 2020).”
Code as Policies: Language Model Programs for Embodied Control (2022, background): “8% P@1 on HumanEval [1], (iii) a new benchmark to evaluate future language models on robotics code-gen problems, and (iv) ablations that analyze how CaP improves metrics of generalization [23] and that it abides by scaling laws – larger models perform better.”
How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (2021, positive): “It is often discussed in the context of out-of-distribution generalization (Hadley, 1994; Hupkes et al., 2020; Keysers et al., 2020; Li et al., 2021), typically relying on synthetic datasets to test models’ compositional abilities (Lake and Baroni, 2018; Kim and Linzen, 2020; McCoy et al., 2020).”
Are Pretrained Language Models Symbolic Reasoners over Knowledge? (2020, background): “Hupkes et al. (2020) study the ability of neural models to capture compositionality.”
Compositionality and Generalization In Emergent Languages (2020, background): “…of neural networks, particularly in linguistic tasks where humans rely on compositionality (e.g., Fodor and Lepore, 2002; Marcus, 2003; van der Velde et al., 2004; Brakel and Frank, 2009; Kottur et al., 2017; Lake and Baroni, 2018; Andreas, 2019; Hupkes et al., 2019; Resnick et al., 2019).”
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation (2020, background): “Transformers have been quickly adopted in practical NLP systems (Storks et al., 2019), but the literature has reported mixed results on the beneﬁt of Transformers over LSTMs in terms of linguistic generalization (Hupkes et al., 2020; van Schijndel et al., 2019).”
The EOS Decision and Length Extrapolation (2020, methodology): “Evidence from a number of settings shows that processing sequences longer than those seen at training time is a challenge for neural models (Dubois et al., 2020; Ruis et al., 2020; Hupkes et al., 2020; Klinger et al., 2020).”