Word lengths are optimized for efficient communication

By S. Piantadosi, Harry J. Tily, Edward Gibson. Proceedings of the National Academy of Sciences of the United States of America, 2011

600 citations recorded by the source corpus.

We demonstrate a substantial improvement on one of the most celebrated empirical laws in the study of language, Zipf's 75-y-old theory that word length is primarily determined by frequency of use. In accord with rational theories of communication, we show across 10 languages that average information content is a much better predictor of word length than frequency. This indicates that human lexicons are efficiently structured for communication by taking into account interword statistical dependencies. Lexical systems result from an optimization of communicative pressures, coding meanings efficiently given the complex statistics of natural language use.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/0b604fcc233483eda6ed6542e3a4e6f389d875f9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Communicative efficiency in language production: Optional case-marking in Japanese (2015, background): “The ideal speaker framework assumes that the pressures for alleviating such encoding difficulties, and perhaps production effort in general, are weighted against a bias for robust information transfer (for related ideas, see Jaeger, 2006; Piantadosi et al., 2011; Zipf, 1949).”
The communicative function of ambiguity in language. (2011, neutral): “Shannon entropy measures the amount of information required on average to disambiguate which meaning in M is intended and is given by where P ( m ) is the probability that meaning m is the intended meaning.”
Typical Decoding for Natural Language Generation (2022, background): “Indeed, there is a long history of studying language via this means (Shannon, 1948, 1951; Hale, 2001; Piantadosi et al., 2011; Pimentel et al., 2020, inter alia ).”
The Goldilocks Effect: Human Infants Allocate Attention to Visual Sequences That Are Neither Too Simple Nor Too Complex (2012, background): “Similar hypotheses about how adults allocate their limited resources in the language domain–for example, those supporting a uniform information principle [33–37]–may suggest that what we have observed in infants reflects a ubiquitous constraint across domains and developmental levels.”
Consequences of the Now-or-Never bottleneck for signed versus spoken languages (2015, methodology): “…interest in rational, rather processing-based, explanations in cognitive science (e.g., Anderson 1990; Chater et al. 2006 Griffiths & Tenenbaum 2009; Oaksford & Chater 1998; 2007; Tenenbaum et al. 2011), including language processing (Gibson et al. 2013; Hale 2001; 2006; Piantadosi et al. 2011).”
Zipf's law of abbreviation as a language universal (2016, neutral): “However, the focus of earlier studies was not on testing the law as a linguistic universal, and the language samples are small (e.g. 7 languages in [1] and [13], 11 languages in [25]).”
Predicting the birth of a spoken word (2015, background): “Number of phonemes is a measure of word length, which has been known since Zipf (43) to be correlated with word frequency [perhaps as a consequence of the evolution of vocabulary to facilitate efficient communication (44)].”