A convergent gambling estimate of the entropy of English

By Thomas M. Cover, R. C. King. 1978

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Theory

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, methodology): “Cover and King (1978) give a lower bound of 1.25 bits per character using a subtle gambling approach.”
On Prediction Using Variable Order Markov Models (2004, background): “Cover and King (1978) considered a very similar prediction game in their well-known work on the estimation of the entropy of English text.”
Revisiting the Predictability of Language: Response Completion in Social Media (2012, background): “An extensive bibliography covering early related work (e.g., insights into the structure of language via information theory, entropy estimates via other techniques and/or for different languages, as well as a broad range of applications of such estimates) can be found in (Cover and King, 1978).”
EXPECTATION IN MELODY: THE INFLUENCE OF CONTEXT AND LEARNING (2006, methodology): “The experimental methodology followed a betting paradigm developed by Cover and King (1978) for estimating the entropy of printed English.”
Predictability , Complexity , and Learning (2002, background): “Cover and King (1978) criticize such observations bynoting that this behavior is impossible in Markov chains of arbitrary order.”
Universal Estimation of Information Measures for Analog Sources (2009, background): “Cover and King [51] later proposed a gambling estimate of English entropy and proved its consistency for stationary ergodic data.”
Minimax Estimation of Functionals of Discrete Distributions (2014, background): “Cover and King [69] gave an overview of the entropy rate estimation literature until 1978.”
Two decades of statistical language modeling: where do we go from here? (2000, unknown): “[11] formulated a gambling setup and used it to derive its own estimate of the entropy of English.”
Handbook of Applied Cryptography (2018, background): “See Shannon [1122] and Cover and King [285] regarding redundancy and Fact 7.”
Limits of Detecting Text Generated by Large-Scale Language Models (2020, background): “Building on a long history of language generation models that are based on statistical knowledge that people have [1]–[6], large-scale, neural network-based language models (LMs) that write paragraph-length text with the coherence of human writing have emerged [7]–[9].”
Predictability, Complexity, and Learning (2000, background): “Cover and King (1978) criticize such observations by noting that this behavior is impossible in Markov chains of arbitrary order.”
An Estimate of an Upper Bound for the Entropy of English (1992, background): “Cover and King (1978) list an extensive bibliography.”