Toward Predicting the Outcome of an A/B Experiment for Search Relevance

42 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Off-policy evaluation for slate recommendation (2016, background): “Previous work on off-policy evaluation and whole-page optimization improves the probability of match between logging and evaluation by restricting attention to small slate spaces [35, 26], introducing assumptions that allow for partial matches between the proposed and observed slates [27], or assumi”
Copeland Dueling Bandits (2015, unknown): “This risk is not merely theoretical.”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “Though effective, the online user evaluation has critical problems: (1) The online interaction between humans and CRSs is slow and usually takes weeks to collect sufficient data to make the assessment statistically significant [93, 52, 212].”
Counterfactual Estimation and Optimization of Click Metrics in Search Engines: A Case Study (2015, background): “It would be interesting to see how to leverage successful ideas in related work [4, 10, 22] to address this issue.”
Returning is Believing: Optimizing Long-term User Engagement in Recommender Systems (2017, background): “And it is proved in [21, 25] that the evaluator is unbiased in the sense that E[v̂ (π )] = v (π ).”