Multi-armed Bandit Algorithms and Empirical Evaluation

By Joannès Vermorel, Mehryar Mohri. 2005

616 citations recorded by the source corpus.

Subfields: Economics and Econometrics, Management Science and Operations Research, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Management Science and Operations Research

In Reinforcement Learning

Citations in the corpus

Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models (2020, methodology): “While the sampling procedure can be optimized by bayesian optimization (Snoek, Larochelle, and Adams 2012) or multi-armed bandit algorithms (Vermorel and Mohri 2005), we choose to keep the method as simple as possible and use random sampling.”
Designing Neural Network Architectures using Reinforcement Learning (2016, background): “However, in the case of large or continuous state spaces, the (cid:15) -greedy strategy of learning has been empirically shown to converge (Vermorel & Mohri, 2005).”
Counterfactual reasoning and learning systems: the example of computational advertising (2012, background): “Even in the simple case of multi-armed bandits, excellent empirical results have been obtained using Thompson sampling (Chapelle and Li, 2011) or fixed strategies (Vermorel and Mohri, 2005; Kuleshov and Precup, 2010).”
Unsupervised label noise modeling and loss correction (2019, methodology): “To ameliorate such instabilities, we propose a decreasing softmax technique (Vermorel & Mohri, 2005) to progressively move from a soft to a hard dynamic bootstrapping.”
SeeDB: Efficient Data-Driven Visualization Recommendations to Support Visual Analytics (2015, methodology): “Our second pruning scheme employs a Multi-Armed Bandit strategy (MAB) [38].”
Automatic ad format selection via contextual bandits (2013, background): “However, [19] also indicates that the family of $ -decreasing algorithms does not have a clear advantage over $ -greedy.”
Explore/Exploit Schemes for Web Content Optimization (2009, methodology): “B-POKER: The POKER scheme [24] is similar to UCB1 but has a different priority function.”
Simulation Studies of Multi-armed Bandits with Covariates (Invited Paper) (2008, methodology): “Greedy selects the action that is expected to yield the highest reward based on the present estimates: αt = arg max αi∈A x(t) β̂i, (5) where x(t) is the vector of covariates, and β̂i is the vector of estimated coefficients.”
An Experts Algorithm for Transfer Learning (2007, methodology): “It has been found that, despite the existence of more sophisticated techniques, a simple ǫ -greedy algorithm, which either chooses the arm that looks the best so far or, with probability ǫ chooses an action at random, was difﬁcult to beat in practice [Vermorel and Mohri, 2005].”
The use of machine learning algorithms in recommender systems: A systematic review (2015, methodology): “Bandit algorithms are used to decide how many times and in which order decisions are made to maximize a given value [65].”
Practical Neural Network Performance Prediction for Early Stopping (2017, methodology): “Using an (cid:15) -greedy exploration strategy [23] and experience replay [12], the agent becomes better with time at discovering designs that obtain high performance on the learning task.”
Ensemble contextual bandits for personalized recommendation (2014, background): “The aforementioned issue is often modeled as a contextual bandit problem [29].”
Quantum Reinforcement Learning (2008, methodology): “One widely used action-selection scheme is -greedy [48], [49], where the best action is selected with probability (1 − ), and a random action is selected with probability ( ∈ (0, 1)).”
Value-Difference Based Exploration: Adaptive Control between Epsilon-Greedy and Softmax (2011, methodology): “Compared to other more complex methods, ε -greedy is often hard to beat [6] and reported to be often the method of ﬁrst choice as stated by Sutton [7].”
Solving two-armed Bernoulli bandit problems using a Bayesian learning automaton (2010, background): “The arm with the most optimistic reward probability estimate is then greedily selected (Vermorel and Mohri, 2005; Kaelbling, 1993).”
Dynamic Pricing and Learning: Historical Origins, Current Research, and New Directions (2013, neutral): “[288]; see further Vermorel and Mohri [289], Cesa-Bianchi and Lugosi [290], and Powell [291].”
Online Learning: A Comprehensive Survey (2018, background): “the multi-armed bandit problem [247, 248, 249, 250, 251, 252, 253] in statistics, is considered a branch of online learning methods where”
Fair lending needs explainable models for responsible recommendation (2018, background): “Multi-armed bandits [8, 58] and other forms of reinforcement learning [31, 38] pose further explainability challenges, due to the statefulness resulting from multiple rounds of learning.”