On the origin of long-range correlations in texts

By Eduardo G. Altmann, Giampaolo Cristadoro, Mirko Degli Esposti. 2012

124 citations recorded by the source corpus.

The complexity of human interactions with social and natural phenomena is mirrored in the way we describe our experiences through natural language. In order to retain and convey such a high dimensional information, the statistical properties of our linguistic output has to be highly correlated in time. An example are the robust observations, still largely not understood, of correlations on arbitrary long scales in literary texts. In this paper we explain how long-range correlations flow from highly structured linguistic levels down to the building blocks of a text (words, letters, etc..). By combining calculations and data analysis we show that correlations take form of a bursty sequence of events once we approach the semantically relevant topics of the text. The mechanisms we identify are fairly general and can be equally applied to other hierarchical settings.

Subfields: Cultural Studies, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cultural Studies

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Scaling Laws for Neural Language Models (2020, background): “This may be a consequence of underlying power-law correlations in language [EP94, ACDE12, LT16], or a more general feature of the model architecture and optimization.”
Languages cool as they expand: Allometric scaling and the decreasing need for new words (2012, background): “Languages, like biological systems demonstrate economies of scale (b < 1) manifesting from a complex dependency structure that mimics a hierarchical “circulatory system” required by the organization of language [33, 41–45] and the limits of the efficiency of the speakers/writers who employ the words”