Variational methods for Reinforcement Learning

By Thomas Furmston, David Barber. 2010

56 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, background): “…such as parsing (Liang et al. 2007; Liang, Jordan, and Klein 2009), grammar induction (Kurihara and Sato 2006; Naseem et al. 2010; Cohen and Smith 2010), models of streaming text (Yogatama et al. 2014), topic modeling (Blei, Ng, and Jordan 2003), and hidden Markov models and part-of-speech…”
Bayesian reasoning and machine learning (2012, unknown): “For this reason and to remain close to our discussion on policy and value iteration, which involved deterministic policies, we focus on this case in our brief discussion here, referring the reader to other texts [80, 292, 105, 106] for the details on the non-deterministic case.”
Black-Box Policy Search with Probabilistic Programs (2015, methodology): “een explored in a number of ways. When the reward is non-negative, variational Bayes and expectation propagation can be used to approximate a posterior in a (non-exponentiated) reward weighted model [Furmston and Barber, 2010]. The exponentiated reward case has similarly been considered [Neumann, 20”
Active inference and epistemic value (2015, background): “This may be useful, because the current scheme offers computational efficiency, by exploiting variational Bayesian techniques (c.f., Furmston & Barber, 2010), accommodates formal constraints on the structure of policies, and comes with a biologically plausible process theory.”
VIREL: A Variational Inference Framework for Reinforcement Learning (2018, background): “Furmston & Barber [19] demonstrate that the posterior over all possible environment models can be integrated over to obtain an optimal policy in a Bayesian setting.”
Making Sense of Reinforcement Learning and Probabilistic Inference (2020, unknown): “For this reason, RL research focuses on computationally efficient approaches that maintain a level of statistical efficiency (Furmston & Barber, 2010; Osband et al., 2017).”
Lifted Stochastic Planning, Belief Propagation and Marginal MAP (2018, background): “Over the last decade multiple authors have introduced explicit reductions showing how stochastic planning can be solved using probabilistic inference (Domshlak and Hoffmann 2006; Toussaint and Storsky 2006; Furmston and Barber 2010; Lang and Toussaint 2010; Furmston and Barber 2011; Neumann 2011; Ko”
Structured Exploration via Hierarchical Variational Policy Networks (2018, methodology): “Variational methods in RL. (Neumann, 2011; Furmston & Barber, 2010) discuss variational approaches for RL problems, but did not consider end-to-end trainable models.”
Relative Entropy Policy Search (2010, methodology): “Single Chain Problem.”
Variational Policy Search via Trajectory Optimization (2013, unknown): “This is the standard formulation for expectation maximization [9], and has been applied to policy optimization in previous work [8, 21, 3, 11].”