Thompson Sampling for Learning Parameterized Markov Decision Processes

By Aditya Gopalan, Shie Mannor. 2014

125 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Why is Posterior Sampling Better than Optimism for Reinforcement Learning? (2016, background): “These results help to suggest that it might be possible to formalize the arguments in our conjecture and maybe even in terms of the problem-speciﬁc frequentist regret [11].”
A Tutorial on Thompson Sampling (2017, background): “Subsequent papers have further developed this theory [43, 44, 45] and have studied extensions of Thompson sampling to reinforcement-learning [35, 46, 47, 48].”
Deep Exploration via Randomized Value Functions (2017, background): “Model learning algorithms exploit generalization in an underlying model of the environment (Kearns and Koller, 1999; AbbasiYadkori and Szepesvári, 2011; Ibrahimi et al., 2012; Ortner and Ryabko, 2012; Osband and Van Roy, 2014a,b; Gopalan and Mannor, 2015).”
Recent advances in reinforcement learning in finance (2021, methodology): “For the Thompson sampling method (Ouyang et al., 2017; Gopalan & Mannor, 2015; Thompson, 1933), the agent keeps track of a belief of the optimal action via a distribution over the set of admissible actions (for each given state).”
Bayesian Optimal Control of Smoothly Parameterized Systems (2015, background): “, Example 1 of Guez et al. (2014)).”
Convex Optimization: Algorithms and Complexity (2015, unknown): “We believe that Thompson sampling style algorithms (e.g., [Russo and Van Roy, 2014b, Gopalan and Mannor, 2015]) can pave the way to efficient algorithms in terms of both sample complexity and computational complexity.”
On the Theory of Reinforcement Learning with Once-per-Episode Feedback (2021, background): “…have been many theoretical results that analyze regret minimization in standard episodic reinforcement (Jaksch et al., 2010; Osband et al., 2013; Gopalan and Mannor, 2015; Osband and Van Roy, 2017; Azar et al., 2017; Jin et al., 2018; Dann et al., 2017; Zanette and Brunskill, 2019; Simchowitz…”
Bandit Algorithms (2020, methodology): “See also the article by Greenewald et al. [2017]. Rafferty et al. [2018] apply Thompson sampling to educational software and note the tradeoff between knowledge and reward. That bandit algorithms have not been used in clinical trials was explicitly noted by Villar et al. [2015]. Microsoft offers a ‘”
Double Thompson Sampling for Dueling Bandits (2016, methodology): “Inspired by the insights from Condorcet dueling bandits, we also discuss how to refine the regret of D-TS in general Copeland dueling bandits.”
Reinforcement Learning with Trajectory Feedback (2020, methodology): “2019) and Thompson-Sampling (or Posterior-Sampling) based algorithms (Osband, Russo, and Van Roy 2013; Gopalan and Mannor 2015; Osband and Van Roy 2017; Russo 2019).”
EventAction (2019, background): “Gopalan and Mannor [23] have extended Thompson sampling to be applicable to MDPs.”
Dueling Posterior Sampling for Preference-Based Reinforcement Learning (2019, background): “Previous work on posterior sampling in RL (Osband et al., 2013; Gopalan and Mannor, 2015; Agrawal and Jia, 2017; Osband and Van Roy, 2017) is focused on learning from absolute rewards, while we extend posterior sampling to both elicit and learn from trajectory-level preference feedback.”