An unbiased offline evaluation of contextual bandit algorithms with generalized linear models

By Lihong Li, Wei Chu, J. Langford, Taesup Moon, Xuanhui Wang. 2011

82 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Machine Learning Methods That Economists Should Know About (2019, methodology): “More recent methods focus on combinations of estimating both the conditional outcome expectations μ(·) and the propensity score e(·) flexibly and combining them in doubly robust methods (Robins and Rotnitzky [1995], Chernozhukov et al.”
Balanced Linear Contextual Bandits (2018, background): “2010), (Dudı́k, Langford, and Li 2011), (Li et al. 2012), (Dudı́k et al.”
Estimation Considerations in Contextual Bandits (2017, background): “There is also a significant body of research that leverages balancing for offline evaluation and learning of contextual bandit or reinforcement learning policies from logged data [57, 28, 44, 27, 43, 59, 37, 60, 10, 38, 64, 26, 39, 58, 66].”
Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning (2022, methodology): “We describe both (i) off-policy counterfactual inference [21, 39] evaluation and (ii) on-policy human (rater) evaluation.”
Online Decision Making with High-Dimensional Covariates (2020, background): “First, our results can be extended from the linear setting to more general function classes such as generalized linear models, which have proven to be useful in several applications (Li et al. 2012).”
Provable Optimal Algorithms for Generalized Linear Contextual Bandits (2017, methodology): “While some UCB-type algorithms using GLMs perform well empirically (Li et al., 2012), there is little theoretical study of them.”
Better Optimism By Bayes: Adaptive Planning with Rich Models (2014, methodology): “(b) Discounted return for TS and α−UCB (with α=1 and upper confidence approximation U0 as defined in (Li et al. 2012)) when using the Bayesian Logistic Regression model, same task setting as (a).”
A Survey on Causal Inference (2020, methodology): “variance and highly relies on the assigned weights. To improve this, similar to the doubly robust method in ATE estimation, doubly robust policy evaluation is proposed in [37]. Later, various methods [8, 16, 64, 76, 134, 135, 137, 155] are proposed to improve those two methods with different setting”
Efficient Counterfactual Learning from Bandit Feedback (2018, unknown): “See Li et al. (2012) and Di-makopoulou, Athey, and Imbens (2017) for possible estimators based on generalized linear models and generalized random forest, respectively.”