An Empirical Evaluation of Thompson Sampling

By Olivier Chapelle, Lihong Li. 2011

1,641 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning (2016, methodology): “It has been demonstrated to compare favorably to UCB1 empirically (Chapelle & Li, 2011).”
Neural Approaches to Conversational AI (2018, positive): “information leads to a natural exploration strategy, inspired by Thompson Sampling (Thompson, 1933; Chapelle and Li, 2012; Russo et al., 2018).”
DRN: A Deep Reinforcement Learning Framework for News Recommendation (2018, methodology): “Our method is significantly different from the MAB group of methods [5, 7, 23, 40, 44, 50] due to its explicit modeling of future rewards, and different from previous MDP methods [27, 35, 36, 38] using user log due to its continuous representation of state and action, and the capability to scale to…”
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2016, background): “The problem of parameter selection has recently received considerable attention in the machine learning community in the context of multi-armed bandit problems [76], [77], [78].”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “Finally, we introduce more recent developments in Section VIII.”
Counterfactual reasoning and learning systems: the example of computational advertising (2012, background): “Even in the simple case of multi-armed bandits, excellent empirical results have been obtained using Thompson sampling (Chapelle and Li, 2011) or fixed strategies (Vermorel and Mohri, 2005; Kuleshov and Precup, 2010).”
Balanced Linear Contextual Bandits (2018, background): “5 , 1 in the Thompson sampling bandits (Chapelle and Li 2011) and among values 1 , 2 , 4 in the UCB bandits (Chapelle and Li 2011).”
Estimation Considerations in Contextual Bandits (2017, background): “5, 1 in the Thompson sampling bandits [22] and among”
Doubly Robust Policy Evaluation and Optimization (2014, background): “The connection suggests a broader applicability of DTR techniques beyond the medical domain, for instance, to the Internet-motivated problems studied in this paper.”
Learning to Learn for Global Optimization of Black Box Functions (2016, neutral): “Here again we encounter a huge variety of strategies, including Thompson sampling [Thompson, 1933, Scott, 2010, Chapelle and Li, 2011, Agrawal and Goyal, 2013], information gain [Villemonteix et al., 2009, Hennig and Schuler, 2012, Russo and Van Roy, 2014, Shahriari et al., 2014, Hern ´…”
Weight Uncertainty in Neural Networks (2015, background): “Let π ∈ [0 , 1] M and =1 π i = 1 , and deﬁne: ) where E M denotes an expectation over the random partitioning of minibatches.”
Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling (2018, background): “Thompson Sampling has been shown to be extremely effective for bandit problems both in practice (Chapelle & Li, 2011; Granmo, 2010) and theory (Agrawal & Goyal, 2012).”
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems (2012, neutral): “In fact the polynomial concentration of R̂n around Rn can be directly derived from our proof of Theorem 2.1.”
(More) Efficient Reinforcement Learning via Posterior Sampling (2013, methodology): “We show through simulation that PSRL significantly outperforms existing algorithms with similar regret bounds.”
Efficient Exploration Through Bayesian Deep Q-Networks (2018, background): “In multi-arm bandit, there are compelling empirical pieces of evidence that Thompson Sampling sometimes provides better results than optimism-under-uncertainty approaches (Chapelle & Li, 2011), while also the performance guarantees are preserved (Agrawal & Goyal, 2012; Russo & Van Roy, 2014a).”
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems (2016, background): “Our paper touches several areas of research, namely Bayesian neural networks, reinforcement learning with deep Q-networks, Thompson Sampling, and dialogue systems.”
A Tutorial on Thompson Sampling (2017, methodology): “[5] proposed using this method to approximate Thompson sampling in a display advertising problem with a logistic regression model of ad-click-through rates.”
EX2: Exploration with Exemplar Models for Deep Reinforcement Learning (2017, methodology): “Other methods avoid tackling the exploration problem directly and use randomness over model parameters to encourage novel behavior (Chapelle & Li, 2011).”