Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract) ∗

By Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar. 2012

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Human-centric Dialog Training via Offline Reinforcement Learning (2020, methodology): “Well-known examples include Trust Region Policy Optimization (TRPO) (Schulman et al., 2015), and use conservative, KL-regularized policy updates to restrict the RL algorithm to stay close to its own prior policy (Haarnoja et al., 2018; Kakade, 2002; Peters et al., 2010; Rawlik et al., 2012).”
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (2018, methodology): “A similar approach is proposed by Rawlik et al. (2013), where a sequence of maximum entropy problems is solved in a Q-learning style framework to eventually arrive at the standard maximum reward solution.”
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018, methodology): “This framework has been used in many contexts, from inverse reinforcement learning (Ziebart et al., 2008) to optimal control (Todorov, 2008; Toussaint, 2009; Rawlik et al., 2012).”
Reinforcement Learning with Deep Energy-Based Policies (2017, methodology): “…(Todorov, 2007), maximum entropy inverse RL (Ziebart et al., 2008), approximate inference using message passing (Toussaint, 2009), Ψ -learning (Rawlik et al., 2012), and G-learning (Fox et al., 2016), as well as more recent proposals in deep RL such as PGQ (O’Donoghue et al., 2016), but…”
Smoothed Dual Embedding Control (2017, background): “, (Rawlik et al., 2012; Nachum et al., 2017a), from a completely different point of views.”
Maximum a Posteriori Policy Optimisation (2018, methodology): “resented here is inspired by a variational inference perspective on RL that has previously been utilised in multiple studies; c.f. Dayan & Hinton (1997); Neumann (2011); Deisenroth et al. (2013); Rawlik et al. (2012); Levine & Koltun (2013); Florensa et al. (2017). Particular attention has b”
Soft Actor-Critic Algorithms and Applications (2018, methodology): “, 2018c), we introduced the Soft Actor-Critic (SAC) algorithm based on the maximum entropy framework (Ziebart et al., 2008; Toussaint, 2009; Rawlik et al., 2012; Fox et al., 2016; Haarnoja et al., 2017).”
Distral: Robust multitask reinforcement learning (2017, methodology): “The distilled policy π 0 can be interpreted as a policy prior, a perspective well-known in the literature on RL as probabilistic inference [28, 13, 21, 8].”
A unified view of entropy-regularized Markov decision processes (2017, methodology): “Similar derivations have been done several times for discounted and episodic MDPs (Littman and Szepesvári, 1996; Ruszczyński, 2010; Azar et al., 2011; Rawlik et al., 2012; Asadi and Littman, 2016; Fox et al., 2016), but we are not aware of any work that considers the average-reward case.”
Latent Space Policies for Hierarchical Reinforcement Learning (2018, background): “We will consider a more general maximum entropy objective (see, e.g. , (Todorov, 2007; Ziebart, 2010; Rawlik et al., 2012; Fox et al., 2016; Haarnoja et al., 2017; 2018)), which augments the objective with the expected entropy of the policy over ρ π ( τ ) : The temperature parameter α > 0…”
Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control (2017, methodology): “The probabilistic trajectories for model-free RL in [40] are obtained by reformulating the stochastic optimal control problem as KL divergence minimisation.”
Information-Theoretic Bounded Rationality (2015, background): “…2012, van den Broek et al., 2010, Friston, 2010, Peters et al., 2010, Tishby and Polani, 2011, Kappen et al., 2012, Neumann and Peters, 2012, Rawlik and Toussaint, 2012, Rubin et al., 2012, Fox and Tishby, 2012, Neumann and Peters, 2013, Grau-Moya et al., 2013, Tishby and Zaslavsky, 2015,…”
Tuning Recurrent Neural Networks with Reinforcement Learning (2016, methodology): “In the next section, we will show how to improve this model with RL.”
Making Sense of Reinforcement Learning and Probabilistic Inference (2020, methodology): “It is possible to view the algorithms of the ‘RL as inference’ approach in this light (Rawlik et al., 2013; Todorov, 2009; Toussaint, 2009; Deisenroth et al., 2013; Fellows et al., 2019); see Levine (2018) for a recent survey.”
Deep Reinforcement Learning amidst Lifelong Non-Stationarity (2020, background): “…lower bound for the evidence O1:T = 1 is log p(O1:T = 1) = logEq [ p(s1:T ,a1:T ,O1:T = 1) q(s1:T ,a1:T ) ] ≥ Eπ [ T∑ t=1 r(st,at)− log π(at|st) ] , which is the maximum entropy RL objective (Ziebart et al., 2008; Toussaint, 2009; Rawlik et al., 2013; Fox et al., 2015; Haarnoja et al., 2017).”
Generating Adversarial Driving Scenarios in High-Fidelity Simulators (2019, background): “Finally, another approach has been inspired by the control-asinference view [18], [19] and adds an explicit policy entropy term to the objective function to encourage exploration [20].”