Bounded-parameter Markov decision processes

By Robert Givan, Sonia M. Leach, Thomas L. Dean. Artificial Intelligence, 2000

353 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

View Paper: https://www.semanticscholar.org/paper/77d864bf4035570566905e98cc787cc033aca431

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

An analysis of model-based Interval Estimation for Markov Decision Processes (2008, background): “The problem of finding a policy that is no worse than the optimal policy for any MDP whose transition function falls in the given confidence interval is interesting but different than our problem and has been well studied (Givan et al., 2000; Nilim and Ghaoui, 2004).”
Percentile Optimization for Markov Decision Processes with Parameter Uncertainty (2010, methodology): “To date, most efforts have focused on the study of robust MDPs (e.g., Nilim and El Ghaoui 2005, Iyengar 2005, Givan et al. 2000, Bagnell et al. 2001).”
Active reinforcement learning (2008, neutral): “Givan’s boundedparameter MDP framework ( Givan et al., 2000 ) allows”
The Importance of Pessimism in Fixed-Dataset Policy Optimization (2020, methodology): “Similar algorithms have been explored in the safe RL literature [8, 21] and the robust MDP literature [9], where algorithms with high-probability performance guarantees are useful in the context of ensuring safety.”
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage (1999, methodology): “There also exist algorithms for computing approximate models [37] and efficient planning algorithms that use these approximate models [58].”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, background): “In such environments, the TPMs are either noisy (Givan et al., 2000; Satia and Lave, 1973; White and Eldeib, 1994) or assumed to change with time (Szita et al., 2002).”
A unifying framework for computational reinforcement learning theory (2009, methodology): “Such a method has success in solving large MDPs with performance guarantees [Givan et al. 2000; Tsitsiklis and Van Roy 1996; Van Roy 2006; Taylor et al. 2009].”
MDPs: Learning in Varying Environments (2002, neutral): “Givan et al. (2000) developed a model similar to ε-MDPs.”
An algebraic approach to abstraction in reinforcement learning (2004, background): “They developed the concept of a Bounded-parameter MDP (BMDP) (Givan et al., 2000) that allows one to model the differences in the block transition probabilities of the aggregated states.”
An empirical evaluation of interval estimation for Markov decision processes (2004, background): “Givan et al. (2000) derived analogous results for a closely related problem where constraints are expressed in a weighted L ∞ norm.”
Approximate Equivalence of Markov Decision Processes (2003, background): “This family can be viewed also as a bounded MDP [ 7 ]; we take a different approach and choose arbitrarily representative MDP for each partition.”
Value Preserving State-Action Abstractions (2020, background): “One approach builds around MDP homomorphisms, introduced by Ravindran and Barto (2002), based on the earlier work of Givan et al. (1997) and Whitt (1978).”
Regret based Robust Solutions for Uncertain Markov Decision Processes (2013, background): “Most of the research in computing robust solutions has assumed a maximin objective, where one computes a policy that maximizes the value corresponding to the worst case realization [8, 4, 3, 1, 7].”