Efficient computation of optimal actions

By E. Todorov. 2009

505 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Taming the Noise in Reinforcement Learning via Soft Updates (2015, methodology): “Other works have used related methods for reinforcement learning [31, 32, 33, 34, 35].”
Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control (2017, methodology): “We now deﬁne the control-Hamiltonian [6, 37, 41] for this control system as This formulation of the control-Hamiltonian is the centre piece of the Pontryagin’s approach to the OCP.”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “13 Control theory concepts also appear to be useful for simplifying optimization problems in certain other settings (Todorov, 2009; Hennequin et al., 2014). learning (Werbos and Si, 2004).”
Bayesian reasoning and machine learning (2012, unknown): “Recently efficient solutions have been developed for classes of MDPs with utilities restricted to KullbackLeibler divergences [152, 278].”
Inverse Optimal Control with Linearly-Solvable MDPs (2010, methodology): “Indeed the recursion used in (Ziebart et al., 2008) to compute the partition function is very similar to the iterative method for computing the desirability function in (Todorov, 2007; 2009b).”
An information-theoretic approach to curiosity-driven reinforcement learning (2012, background): “[28,2]) addresses the problem of finding closed-form solutions to reinforcement learning problems, by reformulating the optimization objective.”
Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models (2014, unknown): “This problem has been addressed using function approximation methods in (Todorov, 2009b).”
Trading Value and Information in MDPs (2012, methodology): “Our approach is related to other algorithms that combine information theoretic functions with optimal control [3, 11, 5], but its setting and scope are different.”
Path Integral Control by Reproducing Kernel Hilbert Space Embedding (2012, background): “Finally, while we focus on ﬁnite horizon problems, path integral formulations for discounted and average cost inﬁnite horizon problems [18], as well as risk sensitive control [2] also exist.”
Making Sense of Reinforcement Learning and Probabilistic Inference (2020, methodology): “It suggests that a generalization of the RL problem can be cast as probabilistic inference through inference over exponentiated rewards, in a continuation of previous work in optimal control (Todorov, 2009).”
Policy gradients in linearly-solvable MDPs (2010, methodology): “We have also shown [16] that the continuous problem (33, 38) is the limit (when  → 0) of continuous-state discrete-time LMDPs constructed via Euler discretization as above.”
Learning variable impedance control (2011, methodology): “continuous state-action spaces, we do not have to learn the value function as it is suggested in [37] via Z-learning.”
Hierarchy Through Composition with Multitask LMDPs (2017, background): “As shown in Todorov (2009b), the Bellman equation in this setting reduces to (I −MiPTi )zi =MiPTb zb (4) where Mi = diag(qi) and, because boundary states are absorbing, zb = qb.”