Entropy and Long-Range Correlations in Literary English
By Werner Ebeling, T. Poeschel. 1993
We investigated long-range correlations in two literary texts, "Moby Dyck" by H. Melville and Grimm's tales. The analysis is based on the calculation of entropylike quantities as the mutual information for pairs of letters and the entropy, the mean uncertainty, per letter. We further estimate the number of different subwords of a given length n. Filtering out the contributions due to the effects of the finite length of the texts, we find correlations ranging to a few hundred letters. Scaling laws for the mutual information (decay with a power law), for the entropy per letter (decay with the inverse square root of n) and for the word numbers (stretched exponential growth with n and with a power law of the text length) were found.
Subfields: Information Theory, Linguistics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Theory
- Marcelo A. Montemurro
- Damián H. Zanette
- Giampaolo Cristadoro
- Dario Amodei
- Eduardo G. Altmann
- Mirko Degli Esposti
In Linguistics
Citations in the corpus
- Scaling Laws for Neural Language Models (2020, background): “This may be a consequence of underlying power-law correlations in language [EP94, ACDE12, LT16], or a more general feature of the model architecture and optimization.”
- On the origin of long-range correlations in texts (2012, methodology): “Apart from these applications, more fundamental extensions of our results should: (i) consider the mutual information and similar entropy-related quantities, which have been widely used to quantify long-range correlations (9, 18) [see (38) for a comparison to correlations]; (ii) go beyond the simple”
- Towards the Quantification of the Semantic Information Encoded in Written Language (2009, background): “In recent years the use of tools drawn from statistical physics has quantitatively revealed rich linguistic structures at many scales, ranging from the domain of syntax to the organisation of whole lexicons and literary corpora [5, 4, 6, 9, 10, 16, 24, 25].”