Planning by Probabilistic Inference

By Hagai Thomas Attias. 2003

308 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, methodology): “y for a non-zero temperature (see Section 2.5) does not in general have the optimal deterministic policy as its mean. Subsequent work expanded further on the connection between control and inference. Attias (2003) proposed to implement a planning algorithm by means of the Baum-Welch-like method in H”
Compress and Control (2014, background): “These ideas have been recently explored in both the neuroscience (Botvinick and Toussaint 2012; Solway and Botvinick 2012) and machine learning (Attias 2003; Poupart, Lang, and Toussaint 2011) literature.”
Training Agents using Upside-Down Reinforcement Learning (2019, background): “There are also conceptual connections to ideas formulated around “RL as inference”, where the solution approach is to infer the values of actions conditioned on the trajectory being optimal [Attias, 2003; Toussaint and Storkey, 2006] (see [Levine, 2018] for a review).”
Active Inference: A Process Theory (2017, background): “This distinction allows actions to be speciﬁed by beliefs about policies, effectively converting an optimal control problem into an optimal inference problem (Attias, 2003; Botvinick & Toussaint, 2012).”
Variational Bayesian learning of nonlinear hidden state-space models for model predictive control (2009, background): “The first application to sequential decision problems [6] emphasised the fact that this methodology links inference and decision making such that algorithmic improvements in one can be used for the other.”
Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. (2012, background): “In view of this, the most intuitive approach in fleshing out our graphical model might be to treat R ˆ as a continuous variable, whose value directly corresponds to reward magnitude or utility (see, e.g., Attias, 2003).”
Active Inference, Curiosity and Insight (2017, background): “This is not only a prerequisite for Bayesian model selection—of the sort we have associated with insight—but is also necessary for planning as inference (Attias, 2003; Botvinick & Toussaint, 2012): the selection of actions under beliefs about their consequences.”
Robot trajectory optimization using approximate inference (2009, methodology): “relation between expectation-maximization and (immediate reward) RL observed by (Dayan & Hinton, 1997), Attias’ (2003) inference approach to planning, and an EM approach to solving MDPs and POMDPs (Toussaint et al.”
Tuning Recurrent Neural Networks with Reinforcement Learning (2016, background): “…the optimal planning in stochastic environments as inference in graphical models, and enables direct application of probabilistic inference techniques such as ExpectationMaximization (EM) and message passing for solving the control problem (Attias, 2003; Toussaint & Storkey, 2006; Toussaint, 2009).”
Understanding the origin of information-seeking exploration in probabilistic objectives for control (2021, background): “…as inference framework (Levine, 2018; Toussaint, 2009a), although existing for almost two decades in the literature (Abdolmaleki et al., 2018; Attias, 2003; K. C. Rawlik, 2013; Theodorou et al., 2010), has recently come to prominence with the state of the art performance of ‘soft’ actor…”
Gaussian Processes in Reinforcement Learning (2003, background): “…could be enforced continuously in the training set. is clear however, that a principled approach to the exploitation vs. exploration tradeoff requires a more faithful representation of value, as was recently proposed in Bayesian Qlearning (Dearden et al. 1998), and see also Attias (2003).”
An Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Reward (2009, methodology): “This formulation has since been applied to operational space control [13] and in the sequential setting it has been studied by [1, 21].”
General duality between optimal control and estimation (2008, methodology): “Researchers in machine learning [1], [14] have used estimation methods to find optimal controls, however these methods operate in the product space of states and controls.”