Sequential Decisions based on Algorithmic Probability

By Marcus Hutter. 2008

628 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Theory

Citations in the corpus

The Arcade Learning Environment: An Evaluation Platform for General Agents (2012, background): “To this end, different theoretical frameworks have been proposed to formalize the notion of “big” artificial intelligence (e.g., Russell, 1997; Hutter, 2005; Legg, 2008).”
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (2017, background): “This approach is particularly convenient when the environment is partially observable (EvenDar, Kakade, & Mansour, 2005; Kearns et al., 1999) or non-Markov (Hutter, 2005).”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, background): “Universal density models such as Solomonoff induction (Hutter, 2005) learn the structure of the state space at a much greater rate than the empirical estimator, violating Assumption 1b.”
Deep learning in neural networks: An overview (2014, background): “This assumption does not hold in the broader fields of Sequential Decision Making and Reinforcement Learning (RL) (Hutter, 2005; Kaelbling, Littman, & Moore, 1996; Sutton & Barto, 1998; Wiering & van Otterlo, 2012) (Section 6).”
PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem (2011, methodology): “The compression drive tends to improve generalization performance, according to the principles of Occam’s Razor and Minimum Description Length (MDL) and Minimum Message Length (MML) [50, 16, 57, 58, 51, 26, 18, 14].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “That is, all data is “holy”, and never discarded, in line with what mathematically optimal general problem solvers should do [109, 237].”
Compress and Control (2014, methodology): “Furthermore, one could attempt to adaptively learn the best discretization (Hutter 2005a) or approximate Equation 4 using Monte Carlo sampling.”
Self-Delimiting Neural Networks (2012, methodology): “…more principled SLIM NNs can learn to actively select in task-speciﬁc ways their own size, that is, their effective number of weights ( = modiﬁable free parameters), in line with the theory of algorithmic probability and optimal universal inductive inference [71, 34, 54, 43, 55, 56, 57, 30].”
Reinforcement Learning with a Corrupted Reward Channel (2017, background): “Previous related works have mainly focused on the wireheading case of Example 3 [Bostrom, 2014; Yampolskiy, 2014], also known as self-delusion [Ring and Orseau, 2011], and reward hacking [Hutter, 2005, p. 239].”
General Video Game Playing (2013, background): “A large portion of the recent work in AGI is based on Hutter’s theoretical framework (9).”
Alignment for Advanced Machine Learning Systems (2020, background): “Dewey (2011) and Hibbard (2012) have attempted to extend the AIXI framework of Hutter (2005) so that it learns a utility function over world-states instead of interpreting a certain portion of its percepts as a reward primitive.”
On Q-learning Convergence for Non-Markov Decision Processes (2018, methodology): “We use the general history-based agent-environment reinforcement learning framework [Hutter, 2005; Hutter, 2016].”
A Neuroevolution Approach to General Atari Game Playing (2014, background): “Broadly, this work is motivated by the idea of general-purpose intelligence [21], [25], applied to the domain of video game playing [37].”
Sequential Constant Size Compressors for Reinforcement Learning (2011, background): “It remains an open problem as how some developmental and general agent may learn to handle Partially Observable Markov Decision Problems (POMDPs) in real-world environments.”