The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions

By J. Urgen Schmidhuber. 2002

142 citations recorded by the source corpus.

Subfields: Atomic and Molecular Physics, and Optics, Computational Theory and Mathematics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atomic and Molecular Physics, and Optics

In Computational Theory and Mathematics

In Statistics and Probability

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “Somemethods explicitly take into account program runtime (Allender, 1992; Schmidhuber, 1997, 2002; Watanabe, 1992); many consider only programs with constant runtime, written in non-universal programming languages (e.g., Hinton & van Camp, 1993; Rissanen, 1986).”
Universal Intelligence: A Definition of Machine Intelligence (2007, methodology): “One solution could be to use Levin’s Kt complexity (Levin 1973), another might be to use Schmidhuber’s Speed prior (Schmidhuber 2002).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “of Kolmogorov complexity that explicitly take into account program runtime [146, 6, 291, 147, 235, 237].”
Optimal Ordered Problem Solver (2002, neutral): “It should also be of interest to study probabilistic Speed Prior -based oops variants (Schmidhuber, 2002e) and to devise applications of oops -like methods as components of universal reinforcement learners (Section 5.3).”
Self-Delimiting Neural Networks (2012, background): “…more principled SLIM NNs can learn to actively select in task-speciﬁc ways their own size, that is, their effective number of weights ( = modiﬁable free parameters), in line with the theory of algorithmic probability and optimal universal inductive inference [71, 34, 54, 43, 55, 56, 57, 30].”
Model-based Utility Functions (2011, background): “However, if prior probabilities of histories are computed using the speed prior (Schmidhuber, 2002) rather than ξ, then the most probable program generating the interaction history can be finitely computed from that history and can be the basis of a computable utility function.”
TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (2022, methodology): “We base our priors on a notion of simplicity, such as stated by Occam’s Razor or the Speed Prior (Schmidhuber, 2002).”
Asymptotic non-learnability of universal agents with computable horizon functions (2013, background): “Other dynamic priors, such as the Speed Prior S [12], have different properties that are not investigated here.”
Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements (2003, background): “The probabilistic environmental reactions are initially u nknown but assumed to be sampled from the axiomatized Speed Prior [43], according to which hard-t o-compute environmental reactions are unlikely.”
Automatic model construction with Gaussian processes (2014, background): “Exhaustive search strategies have been constructed for the space of computable distributions (Hutter, 2002; Levin, 1973; Schmidhuber, 2002), but they remain impractically slow.”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, background): “…an RNN C could learn deterministic policies, taking into account the precise histories after which these policies worked in the past, assuming that what seems random actually may have been computed by some (initially unknown) algorithm, such as a pseudorandom number generator [79, 47, 48, 49, 50].”
Measuring Intelligence through Games (2011, methodology): “One way of doing this would be to rephrase the deﬁnitions of Legg and Hutter in the context of the Speed Prior (Schmidhuber (2002)) instead of Solomonoﬀ’s prior.”
Maximizing Fun by Creating Data with Easily Reducible Subjective Complexity (2013, methodology): “According to the theory of creativity, in both cases, the data is not surprising but boring (Schmidhuber 2002a, 2007c) as it does not allow for further encoding progress—there is no novel pattern.”