A modern Bayesian look at the multi-armed bandit

By Steven L. Scott. 2010

492 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Reinforcement Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Machine Learning Methods That Economists Should Know About (2019, background): “Multi-armed bandit approaches (Scott [2010], Thompson [1933]) attempt to improve over this static design.”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “Finally, we introduce more recent developments in Section VIII.”
Balanced Linear Contextual Bandits (2018, methodology): “Thompson Sampling (Thompson 1933), (Scott 2010), (Agrawal and Goyal 2012), (Russo et al.”
Estimation Considerations in Contextual Bandits (2017, methodology): “Thompson sampling [61, 55, 5, 52] and upper confidence bound (UCB) [40, 12] are two different methods for assigning contexts to arms which are highly effective in dealing with the exploration-exploitation trade-off.”
Learning to Learn for Global Optimization of Black Box Functions (2016, positive): “Here again we encounter a huge variety of strategies, including Thompson sampling [Thompson, 1933, Scott, 2010, Chapelle and Li, 2011, Agrawal and Goyal, 2013], information gain [Villemonteix et al., 2009, Hennig and Schuler, 2012, Russo and Van Roy, 2014, Shahriari et al., 2014, Hern ´…”
Learning to Optimize via Posterior Sampling (2013, background): “Interest in the algorithm grew after empirical studies (Chapelle and Li [15], Scott [34]) demonstrated performance exceeding state-of-the-art methods.”
(More) Efficient Reinforcement Learning via Posterior Sampling (2013, methodology): “We show through simulation that PSRL significantly outperforms existing algorithms with similar regret bounds.”
A Tutorial on Thompson Sampling (2017, background): “This was spurred partly by two influential articles that displayed the algorithm’s strong empirical performance [5, 6].”
Thompson Sampling for Contextual Bandits with Linear Payoffs (2012, background): “…(2010); Graepel et al. (2010); Chapelle & Li (2011); May & Leslie (2011); Kaufmann et al. (2012)) have empirically demonstrated the eﬃcacy of TS: Scott (2010) provides a detailed discussion of probability matching techniques in many general settings along with favorable empirical comparisons…”
Near-Optimal Regret Bounds for Thompson Sampling (2017, background): “This is the first lower bound on the performance of a natural version of Thompson Sampling that is away from the general lower bound of Ω( √ NT ) for the multiarmed bandit problem.”
Dueling Posterior Sampling for Preference-Based Reinforcement Learning (2019, methodology): “A third line of relevant work is posterior sampling for Bayesian logistic regression [12, 16, 37], which is used as our Bayesian credit assignment model.”
Counterfactual Data-Fusion for Online Reinforcement Learners (2017, background): “, [14, 10, 6, 15, 4]), an agent is faced with K ∈ N,K ≥ 2 discrete action choices (often called “arms”), each with its own, independent, and initially unknown reward distribution.”
Selecting Near-Optimal Learners via Incremental Data Allocation (2015, methodology): “In this paper, we develop a novel formulation of what it means to solve the above dual-objective problem, and we present a novel solution approach, inspired by multi-armed bandit literature (Auer, Cesa-Bianchi, and Fischer 2002; Thompson 1933; Scott 2010; Agrawal and Goyal 2012).”
An Information-Theoretic Analysis of Thompson Sampling (2014, background): “A more detailed discussion of the strengths and potential advantages of Thompson sampling can be found in earlier work [10, 15, 24, 26].”
Learning to Learn without Gradient Descent by Gradient Descent (2016, methodology): “We are therefore considering the finite horizon setting that is popular in AB tests (Kohavi et al., 2009; Scott, 2010) and is often studied under the umbrella of best arm identification in the bandits literature (Bubeck et al., 2009; Gabillon et al., 2012).”
Online Learning: A Comprehensive Survey (2018, background): “s of Beta posterior distributions, and is able to achieve the lower bound of Lai and Robbins (1985) for Bernoulli rewards. More other extensive and recent studies of Bayesian Bandits can be found in (Scott, 2010; May et al., 2012; Russo and Van Roy, 2016). 5.3 Adversarial Bandits In the previous set”