A Formal Theory of Inductive Inference. Part II

By Ray J. Solomonoff. 1964

Subfields: Statistical Learning Theory, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

In Information Theory

Citations in the corpus

Concrete Problems in AI Safety (2016, methodology): “However, this could potentially be overcome by employing highly expressive model families such as reproducing kernel Hilbert spaces [70], Turing machines [141, 142], or sufficiently expressive neural nets [62, 77].”
Weight Agnostic Neural Networks (2019, methodology): “Algorithmic Information Theory In AIT [108], the Kolmogorov complexity [56] of a computable object is the minimum length of the program that can compute it.”
Deep learning in neural networks: An overview (2014, neutral): “…of a solution candidate by the length of the shortest program that computes it (e.g., Blumer, Ehrenfeucht, Haussler, & Warmuth, 1987; Chaitin, 1966; Grünwald,Myung, & Pitt, 2005; Kolmogorov, 1965b; Levin, 1973a; Li & Vitányi, 1997; Rissanen, 1986; Solomonoff, 1964, 1978;Wallace & Boulton, 1968).”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, methodology): “s effective size. The compression drive tends to improve generalization performance, according to the principles of Occam’s Razor and Minimum Description Length (MDL) and Minimum Message Length (MML) [50, 16, 57, 58, 51, 26, 18, 14]. On the other hand, the system also continually tries to invent new”
First Experiments with PowerPlay (2012, methodology): “which there exists a path from the online or task inputs to the halt neuron can run self-delimiting programs [10, 3, 26, 21] studied in the theory of Kolmogorov complexity and algorithmic probability [27, 8]. Inspired by a previous architecture [15], neurons other than the inputs and outputs in our ”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, positive): “ Conclusion We introduced novel combinations of a reinforcement learning (RL) controller, C, and an RNN-based predictive world model, M. The most general CMsystems implement principles of algorithmic [263, 130, 147] as opposed to traditional [24, 257] information theory. Here both Mand Care RNNs or ”
One Big Net For Everything (2018, methodology): “As discussed in earlier work [77], according to the Theory of Algorithmic Information (AIT) or Kolmogorov Complexity [84, 31, 6, 34, 85, 36], given some universal computer, U , whose programs are encoded as bit strings, the mutual information between two programs p and q is expressed as K ( q | p )…”