Bayesian Decision Problems and Markov Chains

By P. Randolph. 1968

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Bayesian Adaptive MDPs (Martin, 1967; Duff, 2002) take this a step further by directly modeling uncertainty in the space of task states.”
Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixture (2021, background): “Conditioning on the environment’s uncertainty, a Bayesoptimal policy can set the optimal balance between exploration and exploitation to maximize the return during training [1, 11, 19, 28].”
Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search (2012, methodology): “One formulation of this idea is the augmented Bayes-Adaptive MDP (BAMDP) [19, 9], in which the extra information is the posterior belief distribution over the dynamics, given the data so far observed.”
Design for an Optimal Probe (2003, background): “An optimality equation may be written in terms of the hyperstates (see Martin (1967) for a rigorous Figure 2.”
Bayesian Reinforcement Learning (2010, background): “The book “Bayesian Decision Problems and Markov Chains” by Martin (1967) gives a good overview of the work of that era.”
Percentile Optimization for Markov Decision Processes with Parameter Uncertainty (2010, background): “This example illustrates the need for criteria that address parameter uncertainty in general and specifically in the MDPs (e.g., Ben-Tal and Nemirovski 1998, Silver 1963, Martin 1967, Satia and Lave 1973, Dearden et al. 1999).”
Bayesian sparse sampling for on-line reward optimization (2005, unknown): “Nevertheless, Bayesian approaches have been considered from the outset (Martin, 1967; Bellman, 1961) and interest has re-emerged in this approach (Engel et al., 2003; Dearden et al., 1999; Strens, 2000; Wyatt, 2001).”
A Bayesian Framework for Reinforcement Learning (2000, background): “For infinite horizon problems, the Bayesian optimal solution for action selection requires calculation of an intractable integral over courses of action (Martin, 1967).”
Risk-Averse Bayes-Adaptive Reinforcement Learning (2021, methodology): “Here we describe the Bayesian formulation of decision making in an unknown MDP (Martin, 1967; Duff, 2003).”
VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning (2019, methodology): “In principle, a Bayes-optimal policy can be computed using the framework of Bayes-adaptive Markov decision processes (BAMDPs) (Martin, 1967; Duff & Barto, 2002), in which the agent maintains a belief distribution over possible environments.”
Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes (2005, background): “Since then, the theory of the control of Markov chains has undergone rapid development (e.g. Dynkin 1965, Martin 1967, Heyman and Sobel 1984, Puterman 1990).”