Compositionality and Generalization In Emergent Languages

By Rahma Chaabouni, Evgeny Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, Marco Baroni. 2020

171 citations recorded by the source corpus.

Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositionality. In this paper, we study whether the language emerging in deep multi-agent simulations possesses a similar ability to refer to novel primitive combinations, and whether it accomplishes this feat by strategies akin to human-language compositionality. Equipped with new ways to measure compositionality in emergent languages inspired by disentanglement in representation learning, we establish three main results: First, given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, there is no correlation between the degree of compositionality of an emergent language and its ability to generalize. Third, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmission: The more compositional a language is, the more easily it will be picked up by new learners, even when the latter differ in architecture from the original agents. We conclude that compositionality does not arise from simple generalization pressure, but if an emergent language does chance upon it, it will be more likely to survive and thrive.

Subfields: Artificial Intelligence, Cultural Studies, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cultural Studies

In Natural Language Processing

Citations in the corpus

Experience Grounds Language (2020, background): “testbed for studying language (Werner and Dyer, 1991) and emergent communication (Schlangen, 2019a; Lazaridou et al., 2018; Chaabouni et al., 2020).”
A survey of multi-agent deep reinforcement learning with communication (2022, unknown): “The receiver agent must then either identify the correct goal based on the sender’s signaling [52, 53, 54, 47, 55, 56, 57, 58] or accomplish its single-agent task using the received signals (messages) [59, 60].”
Iterated learning for emergent systematicity in VQA (2021, background): “Several works in the literature have found that compositionality only arises in emergent languages if appropriate environmental pressures are present (Kottur et al., 2017; Choi et al., 2018; Lazaridou et al., 2018; Chaabouni et al., 2020).”
Emergent Communication of Generalizations (2021, background): “The ability to communicate accurately, even on unseen concepts, is not necessarily indicative of more systematic communication; generalization without compositional language is a common ﬁnding in the literature [1, 6, 16, 22].”