Markov Decision Processes

By William T. B. Uther. 2004

1,276 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, methodology): “A semi-Markov decision process (SMDP) (Puterman, 1994) provides a generalized framework, in which the amount of time between two decision points is modeled as a random variable.”
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation (2019, methodology): “…The fact that the density function dπ forms a dual variable of the value function V π is widely known in the optimal control and reinforcement learning literature (e.g., Bertsekas, 2000; Puterman, 2014; de Farias & Van Roy, 2003), and has been leveraged in various works for policy optimization.”
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism (2021, background): “It is well known [38] that a stationary deterministic policy π exists that simultaneously maximizes V (s) for all s ∈ S , and hence maximizing the expected value J(π).”
Continual Learning In Environments With Polynomial Mixing Times (2021, positive): “RL in continuing environments is typically formulated using a ﬁnite, discrete-time, inﬁnite horizon Markov Decision Process (MDP) [6, 5], which is a tuple M = (cid:104)S , A , T, R (cid:105) , where S is the set of states, A is the set of actions, R : S×A → [0 , R max ] is the reward function, and…”
Convex Optimization: Algorithms and Complexity (2015, positive): “The Markov Decision Process (MDP) is a framework for sequential decision-making in Markovian dynamical systems [Bellman, 1957, Puterman, 1994].”
A unified framework for stochastic optimization (2019, methodology): “Derivative-based stochastic optimization began with the semial paper of Robbins and Monro (1951) which launched an entire eld.”
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, methodology): “Deep reinforcement learning (DRL) [147] techniques, such as the Markov decision process (MDP) [148], have recently been used to reformulate path-ﬁnding between entities as a sequential decision-making task.”
Harnessing Inﬁnite-Horizon Off-Policy Evaluation: Double Robustness via Duality (2019, methodology): “Our double robustness for bias reduction is signiﬁcantly different, and instead yields an intriguing connection with the fundamental primal-dual relations between the density (ratio) functions and value functions (e.g., Bertsekas, 2000; Puterman, 2014).”
Optimality Conditions for Inventory Control (2016, background): “The reader may find coverage of these topics in books on MDPs (Bäuerle and Rieder [3], Bertsekas and Shreve [12], Dynkin and Yushkevich [22], Feinberg and Shwartz [29], Heyman and Sobel [42], Hernández-Lerma [38], Hernández-Lerma and Lasserre [40, 41], Puterman [47], Sennott [54]) and on inventory m”
Artificial intelligence in supply chain management: A systematic literature review (2021, positive): “Littman (2001) argues that MDPs make models in sequential, stochastic environments; the nature of the model is that an agent or decision maker inhabits an environment that changes state randomly in response to action choices made by the decision maker.”
Boosted Curriculum Reinforcement Learning (2022, methodology): “1 DEFINITIONS AND NOTATION We recall the notions about Markov Decision Processes (MDPs) (Puterman, 1990), and the notation used for approximate value iteration (AVI) (Farahmand, 2011), that are relevant to our work.”
If MaxEnt RL is the Answer, What is the Question? (2019, background): “The RL objective is: In fully observed MDPs, there always exists a deterministic policy as a solution [39].”