A Contextual Bandit Bake-off
By Alberto Bietti, Alekh Agarwal, J. Langford. 2018
128 citations recorded by the source corpus.
Contextual bandit algorithms are essential for solving many real-world interactive machine learning problems. Despite multiple recent successes on statistically and computationally efficient methods, the practical behavior of these algorithms is still poorly understood. We leverage the availability of large numbers of supervised learning datasets to compare and empirically optimize contextual bandit algorithms, focusing on practical methods that learn by relying on optimization oracles from supervised learning. We find that a recent method (Foster et al., 2018) using optimism under uncertainty works the best overall. A surprisingly close second is a simple greedy baseline that only explores implicitly through the diversity of contexts, followed by a variant of Online Cover (Agarwal et al., 2014) which tends to be more conservative but robust to problem specification by design. Along the way, we also evaluate and improve several internal components of contextual bandit algorithm design. Overall, this is a thorough study and review of contextual bandit methodology.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Susan Athey
- Guido W. Imbens
- Zhengyuan Zhou
- Maria Dimakopoulou
- Aleksandrs Slivkins
- Stefan Wager
- Mihaela van der Schaar
- Ahmed M. Alaa
In Management Science and Operations Research
- Susan Athey
- Guido W. Imbens
- Zhengyuan Zhou
- Maria Dimakopoulou
- Aleksandrs Slivkins
- Stefan Wager
- David A. Hirshberg
- Nathan Kallus
Citations in the corpus
- Balanced Linear Contextual Bandits (2018, methodology): “Adapting a classification task to a bandit problem is a common method for comparing contextual bandit algoriths (Dud´ık, Langford, and Li 2011), (Agarwal et al. 2014), (Bietti, Agarwal, and Langford 2018).”
- Introduction to Multi-Armed Bandits (2019, methodology): “xternally as a Cognitive Service on Microsoft Azure. 4 The contextual bandit algorithms and some of the core functionality is provided via Vowpal Wabbit,5 an open-source library for machine learning. Bietti et al. (2018) compare the empirical performance of various contextual bandit algorithms. Li e”