Three approaches to the quantitative definition of information

By A. Kolmogorov. 1968

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “Given some programming language, the principle of Minimum Description Length (MDL) can be used to measure the complexity of a solution candidate by the length of the shortest program that computes it (e.g., Blumer, Ehrenfeucht, Haussler, & Warmuth, 1987; Chaitin, 1966; Grünwald,Myung, & Pitt, 2005; ”
Causal Inference Using the Algorithmic Markov Condition (2008, neutral): “Recall that the Kolmogorov complexityK(s) of a strings ∈ {0, 1}∗ is defined as the length of the shortest program that generates s u ing a previously defined universal prefix Turing machine [17], [18], [19], [20], [21], [4], [22].”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, methodology): “The compression drive tends to improve generalization performance, according to the principles of Occam’s Razor and Minimum Description Length (MDL) and Minimum Message Length (MML) [50, 16, 57, 58, 51, 26, 18, 14].”
First Experiments with PowerPlay (2012, background): “Thus, any network topology in which there exists a path from the online or task inputs to the halt neuron can run self-delimiting programs [10, 3, 26, 21] studied in the theory of Kolmogorov complexity and algorithmic probability [27, 8].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “Given some RL problem, according to the AIT argument (Sec.”
One Big Net For Everything (2018, methodology): “As discussed in earlier work [77], according to the Theory of Algorithmic Information (AIT) or Kolmogorov Complexity [84, 31, 6, 34, 85, 36], given some universal computer, U , whose programs are encoded as bit strings, the mutual information between two programs p and q is expressed as K ( q | p )…”
Occam's razor is insufficient to infer the preferences of irrational agents (2017, methodology): “Our simplicity measure is minimum description length of an object, deﬁned as Kolmogorov complexity [Kolmogorov, 1965], the length of the shortest program that outputs a string describing the object.”
Self-Delimiting Neural Networks (2012, methodology): “…more principled SLIM NNs can learn to actively select in task-speciﬁc ways their own size, that is, their effective number of weights ( = modiﬁable free parameters), in line with the theory of algorithmic probability and optimal universal inductive inference [71, 34, 54, 43, 55, 56, 57, 30].”
Algorithmic statistics (2000, unknown): “Probabilistic Statistics: In ordinary statistical theory one proceeds as follows, see for example [5]: Suppose two discrete random variables X, Y have a joint probability mass function p ( x, y ) and marginal probability mass functions p 1 ( x ) = y p ( x, y ) and p 2 ( y ) = x p ( x, y ).”
An entropy-based approach for measuring complexity in supply chains (2010, methodology): “Kolmogorov’s entropy (Kolmogorov 1965) or Reny’s entropy can be used as a measure of uncertainty as well and they will be studied in the future work.”
Simplicity versus likelihood in visual perception: from surprisals to precisals. (2000, background): “Independently, this theorem has been proved too by the other two founding fathers of AIT, Kolmogorov (1965) and Chaitin (1969).”