Dynamic Programming and Markov Processes

By R. Howard. 1960

Subfields: Game & Decision Theory, Algorithms & Complexity

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Game & Decision Theory

In Algorithms & Complexity

Citations in the corpus

Learning World Graphs to Accelerate Hierarchical Reinforcement Learning (2019, methodology): “World graph traversal is inspired by classic optimal planning in Markov Decision Processes with dynamic programming [9, 27, 91, 85].”
Striving for Simplicity in Off-policy Deep Reinforcement Learning (2019, methodology): “This enables re-using the experience collected by any policy to improve the value estimates under the optimal policy, as utilized by value iteration [23, 7] and Q-learning [61].”
Deep Reinforcement Learning and the Deadly Triad (2018, methodology): “It also allows an efficient form of policy improvement (Howard, 1960) by directly estimating the values of a policy that is greedy with respect to current value estimates, as used in value iteration (Bellman, 1957; Howard, 1960; Puterman, 1994) and Q-learning (Watkins, 1989; Watkins and Dayan, 1992)”
Reinforcement Learning: A Survey (1996, background): “There are many good references toMDP models (Bellman, 1957; Bertsekas, 1987; Howard, 1960; Puterman, 1994).”
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition (1999, background): “Kaelbling (1993) was the first to point out that a value function learned from a hierarchical policy could be evaluated incrementally to yield a potentially much better non-hierarchical policy.”
Multiagent learning using a variable learning rate (2002, background): “Markov decision processes A Markov decision process (MDP) [3,13] is a tuple, ( S , A ,T,R) , where S is the set of states, is the set of actions, T is a transition function 0 , 1 , and R is a S × A → R deﬁnes the reward received when selecting an action from the given state.”
An MDP-Based Recommender System (2002, neutral): “Various exact and approximate algorithms exist for computing a policy, and the best known are policy-iteration (Howard, 1960) and value-iteration (Bellman, 1962).”
Compress and Control (2014, methodology): “4 to cover the case of on-policy control or policy iteration (Howard 1960) would be highly desirable.”
A unified view of entropy-regularized Markov decision processes (2017, neutral): “In known environments, this optimization problem has been studied (at least) since the inﬂuential work of Bellman (1957) and Howard (1960): the optimal behavior policy can be formulated as the solution of the Bellman optimality equations.”
Hierarchical Solution of Markov Decision Processes using Macro-actions (1998, background): “Markov decision processes (MDPs) [11, 22] have proven tremendously useful as models of stochastic planning and decision problems.”
Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients (2017, methodology): “Policy iteration was adopted to learn the optimal policy π∗ for optimal glycemic trajectories [Howard, 1960].”
Planning, Learning and Coordination in Multiagent Decision Processes (1996, methodology): “We consider DTP problems that can be modeled as completely observable MDPs [26, 42].”
Policy-Gradient Algorithms for Partially Observable Markov Decision Processes (2003, background): “5) is the basis of value iteration [Howard, 1960].”
Dynamic Optimization of Long‐Term Growth Rate for a Portfolio with Transaction Costs and Logarithmic Utility (2001, methodology): “…1990b; Akian et al. 1996 for the elliptic case). dynamic optimization of long-term growth rate 185 The Howard algorithm that we consider here (see Howard 1960) consists of an iteration algorithm on the control and value functions, starting from a0 or (U0, π0): for n ≥ 1 an(z) ∈ argmax a∈A(z)…”
Markov Decision Processes with Applications to Finance (2011, background): “I Shapley (1953) I Bellman (1957, Reprint 2003) I Howard (1960) I Bertsekas and Shreve (1978) I Puterman (1994) I Hernández-Lerma and Lasserre (1996) I Bertsekas (2001, 2005) I Feinberg and Shwartz (2002) I Powell (2007) I B and Rieder (2011)”
Reinforcement learning in the brain (2009, methodology): “…convergence of ‘policy iteration’ methods (in which a policy is iteratively improved through bootstrapping of the values derived given each policy; Howard (1960) and Bertsekas and Tsitsiklis (1996)) ensure that if the proper conditions on the learning rate are met and all state-action pairs are…”
Modified Policy Iteration Algorithms for Discounted Markov Decision Problems (1978, methodology): “Howard [9]) these algorithms are equivalent to performing the policy evaluation step by a fixed number of successive approximation iterations.”