Reusing historical interaction data for faster online learning to rank for IR

By Katja Hofmann, Anne Schuth, Shimon Whiteson, Maarten de Rijke. 2013

122 citations recorded by the source corpus.

Subfields: Computer Science Applications, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Information Systems

In Management Science and Operations Research

Citations in the corpus

DRN: A Deep Reinforcement Learning Framework for News Recommendation (2018, methodology): “Therefore, rather than doing random exploration, we apply a Dueling Bandit Gradient Descent algorithm [16, 17, 49] to do the exploration.”
Toward Predicting the Outcome of an A/B Experiment for Search Relevance (2015, background): “Recent, some authors applied oﬄine evaluation techniques to estimate online metrics for a speciﬁc type of experiments, namely interleaving [12, 13].”
Counterfactual Evaluation and Learning for Search, Recommendation and Ad Placement (2016, background): “…(also called estimation of treatment eﬀects and oﬀ-policy evaluation) and counterfactual learning (also called policy optimization) has made rapid progress in recent years [8, 3, 7, 6, 2, 13, 22, 9, 11, 14, 15, 19, 24, 12, 18], and it is starting to have impact in commercial applications.”
Preference-based Online Learning with Dueling Bandits: A Survey (2018, background): “In the context of information retrieval, Hofmann et al. (2013) investigate whether and how previously collected historical interaction data can be used to speed up online learning to rank.”
Fidelity, Soundness, and Efficiency of Interleaved Comparison Methods (2013, methodology): “If all comparisons are done with live data, then applications such as learning to rank [Hofmann et al. 2013], which perform many comparisons, need prohibitive amounts of data.”
Batch learning from logged bandit feedback through counterfactual risk minimization (2015, background): “…multi-armed bandits (Shivaswamy and Joachims, 2012) and contextual bandits (Strehl et al., 2010), pre-selecting retrieval functions for search engines (Hofmann et al., 2013), policy evaluation for contextual bandits (Li et al., 2011), and reinforcement learning (Thomas et al., 2015) to name a few.”
Bias and Debias in Recommender System: A Survey and Future Directions (2020, background): “A simple yet effective solution to estimate a position-based propensity model is result randomization, where the ranking results are shuffled randomly and collect user clicks on different positions to compute propensities scores [5], [95], [96], [97], [98].”
Unbiased Learning-to-Rank with Biased Feedback (2016, methodology): “Fourth, allowing the LTR algorithm to randomize what is presented to the user, like in online learning algorithms [17, 6] and batch learning from bandit feedback (BLBF) [25] can overcome the problem of bias in click data in a principled manner.”
Counterfactual Estimation and Optimization of Click Metrics in Search Engines: A Case Study (2015, background): “An interesting exception is the recent work [13] that applies unbiased ofﬂine evaluation to infer interleaving outcomes.”
Counterfactual Risk Minimization (2015, background): “…several applications that require learning from logged ban-dit feedback data: warm-starting multi-armed bandits (Shivaswamy & Joachims, 2012), pre-selecting retrieval functions for search engines (Hofmann et al., 2013), and policy evaluation for contextual bandits ( Li et al., 2011), to name a few.”
Click Models for Web Search (2015, methodology): “e assignment of the attractiveness parameters according to [Hofmann et al., 2013] is given in the Table 8.”