Sample-efficient Nonstationary Policy Evaluation for Contextual Bandits
By Miroslav Dud'ik, Dumitru Erhan, J. Langford, Lihong Li. 2012
41 citations recorded by the source corpus.
We present and prove properties of a new offline policy evaluator for an exploration learning setting which is superior to previous evaluators. In particular, it simultaneously and correctly incorporates techniques from importance weighting, doubly robust evaluation, and nonstationary policy evaluation approaches. In addition, our approach allows generating longer histories by careful control of a bias-variance tradeoff, and further decreases variance by incorporating information about randomness of the target policy. Empirical evidence from synthetic and realworld exploration learning problems shows the new evaluator successfully unifies previous approaches and uses information an order of magnitude more efficiently.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Lihong Li
- Alekh Agarwal
- Yoshua Bengio
- Pieter Abbeel
- Thorsten Joachims
- Ruslan Salakhutdinov
- Jan Peters
In Management Science and Operations Research
- Alekh Agarwal
- Lihong Li
- Thorsten Joachims
- J. Langford
- Adith Swaminathan
- Jan Peters
- Alex Slivkins
- L. Eon Bottou
Citations in the corpus
- Counterfactual reasoning and learning systems: the example of computational advertising (2012, methodology): “This variance reduction technique is in fact identical to the doubly robust contextual bandit evaluation technique of Dudík et al. (2012).”
- Making Contextual Decisions with Low Technical Debt (2016, methodology): “Our ML methodology guarantees these experiments are counterfactually accurate [17, 19, 33, 34].”