Bayesian Counterfactual Risk Minimization
By Ben London, Ted Sandler. 2018
39 citations recorded by the source corpus.
We present a Bayesian view of counterfactual risk minimization (CRM) for offline learning from logged bandit feedback. Using PAC-Bayesian analysis, we derive a new generalization bound for the truncated inverse propensity score estimator. We apply the bound to a class of Bayesian policies, which motivates a novel, potentially data-dependent, regularization technique for CRM. Experimental results indicate that this technique outperforms standard $L_2$ regularization, and that it is competitive with variance regularization while being both simpler to implement and more computationally efficient.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Noveen Sachdeva
- Yi-Hsun Su
- Thorsten Joachims
- Sergey Levine
- Andrew McCallum
- Yoshua Bengio
- Percy Liang
- Tong Zhang
Citations in the corpus
- Off-policy Bandits with Deficient Support (2020, methodology): “With terabytes of log data readily available in many online systems, a range of algorithms has been proposed for batch learning from such logged contextual-bandit feedback [5, 6, 20–22, 24, 26].”