On the Theory of Dynamic Programming.

By Richard Bellman. 1952

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Unmasking Clever Hans predictors and assessing what machines really learn (2019, methodology): “[5] make use of the fact that the optimal Q-function, Q∗ must obey the Bellman equation [115]”
Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications (2018, unknown): “ory of optimal control [5] including the Bellman equation and Markov decision process (MDP) together with TD learning to form a well-known Q-learning.”
Active Inference, Curiosity and Insight (2017, methodology): “This precludes solutions based on the Bellman optimality principle (Bellman, 1952).”
Reinforcement Learning for Combinatorial Optimization: A Survey (2020, methodology): “…maximum of state-action value functions across all policies: One of the fundamental equations used for solving MDPs is called the Bellman equation [Bellman, 1952] and has the following form: where T ( s, a, s ′ ) is the transition function, representing the probability of transitioning from s to…”
Systematic Rectification of Language Models via Dead-end Analysis (2023, background): “For any MDP, Q ∗ is unique (Bertsekas & Tsitsiklis, 1996) and can be found as the fixed point of the Bellman operator (Bellman, 1954; 1957).”
Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective (2023, background): “Focusing on two basic mathematical problems as well as Dynamic Programming, we show that a bounded-depth Transformer without CoT struggles with these tasks unless its size grows prohibitively large.”
Sharing Knowledge in Multi-Task Deep Reinforcement Learning (2020, background): “, the ones that satisfy the Bellman optimality equation (Bellman, 1954): Q∗(s, a) , ∫ S P(s ′|s, a) [R(s, a, s′) + γmaxa′ Q∗(s′, a′)] ds′.”
Meta-learning of Sequential Strategies (2019, methodology): “Bayes-optimal sequential decision-making is the decision strategy that follows from the theory of subjective expected utility [Savage, 1972] and the method of dynamic programming [Bellman, 1954].”
Batch Informed Trees (BIT*): Sampling-based optimal planning via the heuristically guided search of implicit random geometric graphs (2014, methodology): “It can be viewed as an extension of incremental graphsearch techniques, such as Lifelong Planning A* (LPA*), to continuous problem domains as well as a generalization of existing sampling-based optimal planners.”
Reinforcement Learning or Active Inference? (2009, methodology): “This is seen in cognitive psychology, through the use of reinforcement learning models like the Rescorla-Wagner model [1]; in computational neuroscience and machine-learning as variants of dynamic programming, such as temporal difference learning [2–7] and in economics, as expected utility theory [8”
Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges (2015, methodology): “3), according to Bellman’s principle of optimality (Bellman, 1952), is such that for every t = 0,1, .”
A uniform Tauberian theorem in optimal control (2010, background): “Finite horizon problem of optimal control have been studied intensively since the pioneer work of Stekhov, Pontryagin, Boltyanskii [28], Hestenes [18], Bellman [9] and Isaacs [19,20] during the cold war - see for instance [7,23,24] for major references, or [14] for a short, clear introduction.”