Deep Learning with Logged Bandit Feedback

By Thorsten Joachims, Adith Swaminathan, Maarten de Rijke. 2018

161 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback (2018, neutral): “Our contribution is to modify the self-normalizing estimator (Kong, 1992; Precup et al., 2000; Swaminathan and Joachims, 2015b; Joachims et al., 2018) to be applicable to neural networks.”
Off-policy Bandits with Deficient Support (2020, methodology): “In particular, the form of the inner objective coincides with that of BanditNet [11], which is known to work well for deep network training by controlling propensity overfitting [24].”
Bayesian Counterfactual Risk Minimization (2018, methodology): “Following prior work [2, 32, 14], we use a standard supervised-to-bandit conversion to simulate logged bandit feedback.”
Harnessing multimodal data integration to advance precision oncology (2021, methodology): “5) would learn policies to recommend future therapies for new patients on the basis of historical patient records of administered treatments, patient contexts (for example, a pretreatment CT scan and H&E-stained biopsy sample) and survival outcomes 101,102 .”
CAB: Continuous Adaptive Blending for Policy Evaluation and Learning (2019, methodology): “, 2017), CAB is sub-differentiable which allows its use as the training objective in off-policy learning algorithms like POEM (Swaminathan & Joachims, 2015a) and BanditNet (Joachims et al., 2018).”
Imitation-Regularized Offline Learning (2019, methodology): “Our solution connects to other variance reduction techniques [31, 14] and we show that ours are more stable with varying hyperparameter choices.”
What is the Effect of Importance Weighting in Deep Learning? (2018, methodology): “For example, Joachims et al. [2018] use the technique to learn from logged contextual bandit feedback.”
Causal embeddings for recommendation (2017, methodology): “The most recent work in the BLBF setup introduces BanditNet, an algorithm for training deep neural networks via logged contextual bandit feedback data [12].”
Seq2Slate: Re-ranking and Slate Optimization with RNNs (2018, background): “Finally, an interesting future direction is to study off-policy correction for seq2slate [Joachims et al., 2018, Chen et al., 2019].”